AnswersScale & Architecture

Does Quantization Hurt Accuracy?

How do production systems use aggressive quantization for speed without letting retrieval accuracy crater?

2 min readJuly 2026

The short answer

Yes, quantization loses information, but the loss is recoverable if you spend it where rankings aren't decided. The production pattern is two-stage: use aggressively quantized vectors (int8, product quantization, even 1-bit binary) to select candidates fast and cheap, then rescore that shortlist with full-precision vectors.

Quantization's ranking mistakes concentrate in the long tail of near-ties, not in the clear top results, so rescoring recovers nearly all the quality while the memory and speed savings stay.

Where does the accuracy actually go?

Quantization compresses the space between vectors, so documents with genuinely different similarities start to tie. For the clear winners, the passages obviously close to the query, the ordering survives even harsh compression. For the marginal candidates ranked 50–200, ties break randomly. Naively returning quantized scores as final rankings surfaces that tail noise; using them only to build a candidate set hides it.

Why does rescoring fix it?

Rescoring works because the candidate set only needs to contain the right documents, not order them. If the true top-10 reliably lands somewhere in the quantized top-100, exact rescoring of 100 vectors (trivial compute) reconstructs the true order. You keep quantized speed on the expensive full-corpus scan and pay full precision only on a shortlist.

Spend the accuracy loss where rankings are not decided.

Whole corpus

quantized scan, fast

Candidate top-100

right docs present, order noisy

Rescore at full precision

True top-10

How far can you push it?

You can push surprisingly far: binary quantization stores one bit per dimension (a 32× reduction over float32), and Hamming distance over binary codes is fast enough to scan huge candidate sets, with rescoring cleaning up the ranking. TopK's production kernel makes the point concrete: hand-optimized ARM NEON code sustains nearly 350 GB/s of Hamming-distance throughput (measured July 2025, ~15× the scalar baseline on a single thread), which is what makes brute-force scanning of binary candidates viable at billion scale.

When should you be careful?

Measure recall on your data before trusting any compression level: out-of-distribution embeddings and very anisotropic spaces quantize worse than benchmark datasets suggest, and the fix (a larger candidate set) costs some of the speed back. The knob is candidate-set size; tune it against a fixed recall target, not by feel.

Keep reading

Is Multi-Vector Retrieval Too Expensive?It was: token-level embeddings cost 10–100× more storage. Two-stage designs and MaxSim approximations like SMVE now make it a viable first stage.Why Object Storage for Vector Search?At billions of vectors the dominant cost is keeping data hot. Object storage decouples storage from compute, so cost tracks query load, not data size.How Do You Benchmark Vector Search?Benchmark your own workload: mixed read/write load, your filters, and latency percentiles at a fixed recall target. Read-only QPS flatters every engine.