Source: https://www.astrmira.com/en/research/papers/contextual-quantization/
Language: en

[Home](https://www.astrmira.com/en/)/[Research](https://www.astrmira.com/en/research/)/2609.09867

LOW-BIT LLMS / ARXIV:2609.09867

# Contextual Utility of Quantization Moves in Extreme Low-Bit LLMs

Studies the real utility of a discrete move in extreme low-bit LLMs by evaluating the move along its own path and combining decisions in an evolving model state.

[Read on arXiv](https://arxiv.org/abs/2609.09867)[Open versioned PDF](https://arxiv.org/pdf/2609.09867v1)

Paper details

**Authors**: Wenxuan Xiao, Xu Cao

**First submitted**: 2026-09-09

**Latest revision**: 2026-09-09

**Public version**: arXiv 2609.09867v1

LLM quantization · Post-training · Discrete optimization

A move is shaped by the path it takes and by the state produced by earlier moves.

## A better reconstruction is not always a better model

Post-training quantization changes a finite code to reduce weight or activation reconstruction error, while the model is ultimately judged by predictive loss and task performance. The paper asks how to tell whether a legal code move is actually useful at a fixed bit width.

Utility depends on the displacement of the move. A gradient evaluated only at the start misses curvature along the path; evaluating at the move midpoint gives a better sign estimate for endpoint loss. Low-bit Llama-3.2 experiments test this distinction.

## Quantization moves interact

Enumeration over legal quantized states shows that combined utility is approximately quadratic. Small pairwise interactions can still change the best multi-objective tradeoff, and the same move can switch from helpful to harmful in a different state.

Midpoint readings repair local choices. For larger combinations, the method reevaluates from states that are actually reached and verifies exact endpoints; endpoint-evaluated beam search finds sparse updates that outperform larger one-shot changes.

## From static scores to state-aware optimization

The work moves quantization optimization to the level of individual code changes: define the objective, evaluate each move along its path, and recombine moves as the state evolves. Experiments connect these choices to task accuracy and held-out perplexity.

Read and cite

## Continue with the original paper.

The complete derivations, experimental setup, and results are available in the public manuscript.

[Abstract and version history ↗](https://arxiv.org/abs/2609.09867)  
[Full paper PDF ↗](https://arxiv.org/pdf/2609.09867v1)

Related papers

VECTOR SEARCH · 2026.09

### When do compressed vectors make the same decisions?

Research at a glance

When Does Low-Bit Quantization Preserve the Decisions of Vector Search?

Places quantization error inside the ranking and graph-pruning decisions that vector search actually makes, relating reliability to comparison margins, correlated residuals, and execution traces.

Vector search · Decision stability

[Explore this research](https://www.astrmira.com/en/research/papers/low-bit-decisions/)

[arXiv](https://arxiv.org/abs/2609.09854)

REPRESENTATION GEOMETRY · 2026.05

### What survives at one or two bits?

Research at a glance

Covariance Structure and Coordinate Heterogeneity Govern Binary Quantization of Contrastive Embeddings

Explains low-bit ranking behavior through covariance structure and coordinate heterogeneity, including why an extra magnitude bit and random rotation help different representations in different ways.

Representation geometry · Binary quantization

[Explore this research](https://www.astrmira.com/en/research/papers/covariance-binary-quantization/)

[arXiv](https://arxiv.org/abs/2605.17524)