Source: https://www.astrmira.com/en/research/papers/covariance-binary-quantization/
Language: en

[Home](https://www.astrmira.com/en/)/[Research](https://www.astrmira.com/en/research/)/2605.17524

REPRESENTATION GEOMETRY / ARXIV:2605.17524

# Covariance Structure and Coordinate Heterogeneity Govern Binary Quantization of Contrastive Embeddings

Explains low-bit ranking behavior through covariance structure and coordinate heterogeneity, including why an extra magnitude bit and random rotation help different representations in different ways.

[Read on arXiv](https://arxiv.org/abs/2605.17524)[Open versioned PDF](https://arxiv.org/pdf/2605.17524v2)

Paper details

**Authors**: Wenxuan Xiao

**First submitted**: 2026-05-17

**Latest revision**: 2026-05-29

**Public version**: arXiv 2605.17524v2

Representation geometry · Binary quantization · Covariance

Coordinate scale and cross-coordinate structure jointly determine the ranking information that survives compression.

## Why can opposite designs both work?

Some binary quantization systems randomly rotate vectors while others preserve the original axes, and retrieval quality varies widely across representations at the same bit width. The paper studies these observations through the statistics of contrastive embeddings.

## Separating covariance from coordinate heterogeneity

Under a Gaussian model, the full covariance structure affects ranking fidelity; marginal variances alone miss accumulated cross-coordinate signal. Unequal coordinate variances determine how much a magnitude bit can add and whether random rotation helps or removes useful structure.

Rotation equalizes variance and can support isotropic distance correction, but it can also erase heterogeneity used by another code. The paper derives approximate fidelity expressions and empirical scaling relationships across models and dimensions.

## Turning representation structure into a design signal

Experiments across nine embedding families and eighteen datasets examine covariance, magnitude bits, and rotation. The results offer statistical guidance for choosing an encoding and preprocessing strategy under the stated Gaussian and approximation conditions.

Read and cite

## Continue with the original paper.

The complete derivations, experimental setup, and results are available in the public manuscript.

[Abstract and version history ↗](https://arxiv.org/abs/2605.17524)  
[Full paper PDF ↗](https://arxiv.org/pdf/2605.17524v2)

Related papers

VECTOR SEARCH · 2026.09

### When do compressed vectors make the same decisions?

Research at a glance

When Does Low-Bit Quantization Preserve the Decisions of Vector Search?

Places quantization error inside the ranking and graph-pruning decisions that vector search actually makes, relating reliability to comparison margins, correlated residuals, and execution traces.

Vector search · Decision stability

[Explore this research](https://www.astrmira.com/en/research/papers/low-bit-decisions/)

[arXiv](https://arxiv.org/abs/2609.09854)

SEARCH SYSTEMS · 2026.05

### QuIVer: building the search graph directly in binary space

Research at a glance

QuIVer: Rethinking ANN Graph Topology via Training-Free Binary Quantization

Builds, prunes, and navigates an ANN graph in a training-free two-bit space, reading full vectors only for final reranking and exposing the data regimes where compact topology works.

Approximate nearest neighbors · Graph indexes

[Explore this research](https://www.astrmira.com/en/research/papers/quiver/)

[arXiv](https://arxiv.org/abs/2605.02171)