AnswersScale & Architecture

Which Recall Number Actually Matters?

Vector databases advertise near-perfect recall while RAG answers still come back wrong, so which recall number should you evaluate before you commit?

3 min readJuly 2026

Two different numbers share the name "recall," and they answer different questions. Index recall measures whether an approximate index reproduces exhaustive search over the same vectors, which is a fidelity claim about infrastructure. Retrieval quality measures whether the correct document surfaces at all, judged against human relevance labels, which is a claim about your answers.

A vector database can score 99% on the first while your product fails the second, because the embedding sets the quality ceiling and the index only preserves it. Evaluate retrieval quality first; it decides whether the answer is right. Check index recall second; it decides whether the infrastructure wastes any of the quality you have.

What does index recall measure?

Index recall compares an approximate nearest-neighbor index against brute-force search over the same embeddings: of the true top-10 nearest vectors, how many did the index return? It is the standard axis of ANN-Benchmarks, and it is the number most vector databases advertise, because it measures the thing an index actually controls. A database advertising 99% recall@10 is making exactly this claim: our approximation is faithful to exhaustive search.

That claim is honest and useful. It says nothing about your answers.

What does retrieval quality measure?

Retrieval quality compares search results against human relevance judgments: of the documents a person would call correct for this query, how many surfaced? Benchmarks like BEIR and ViDoRe measure this form, reported as recall@k or nDCG@k against labeled ground truth. The denominator is what is true, not what exhaustive search over your embeddings would have returned.

Index recall grades the last hop; the answer was decided a hop earlier.

Query

Embedding model

the quality ceiling is set here

ANN index

index recall is measured here

Top-k results

Why can both numbers be true while the answer is wrong?

The index can only find what the embedding preserved. If a single-vector embedding averaged the table row away, exhaustive search misses it too, and a 100%-recall index faithfully retrieves the wrong pages. This is the confusing moment in a vendor bake-off: two systems both report 99% recall, and their answers differ wildly, because the number never measured the answers.

The scale of the ceiling effect is measurable. On ViDoRe v3 (June 2026), TopK's compact late-interaction model beat a dense embedding model 80× its size by +34% recall and +30% nDCG@10 on average. No index setting closes a gap like that, because the gap lives in the representation, not the approximation.

Which should you evaluate?

Evaluate both, in this order, and insist they are labeled. Measure retrieval quality first, on your own documents and queries, because it decides whether the product answers correctly. Check index recall second, at a fixed quality level, because it tells you what the approximation costs (how to benchmark).

TopK publishes both forms and labels them. Its topk-bench harness measures recall@10 against precomputed exact nearest neighbors, which is index recall. Its semantic_index evaluation reports 52.88% nDCG@10 end to end across BEIR against relevance labels, within about 1% of exhaustive MaxSim, which is retrieval quality plus the fidelity cost, stated separately. That is the reporting pattern to demand from any vendor.