What Is Multi-Vector Retrieval?

Why isn't one embedding per document enough?

A single embedding is a lossy summary. Compressing a whole document into one ~1,000-dimensional vector keeps the broad topic and discards specifics: exact figures, rare terms, a definition buried in one clause. Queries that depend on those specifics land far from the document's summary vector, and the right passage never surfaces.

This is a capacity limit, not a training gap: a 2025 result shows that fixed-size single vectors provably cannot represent all the combinations of documents that may be jointly relevant to a query.

How does it work?

1. Encode each document into one embedding per token and store all of them.
2. Encode the query the same way.
3. Score with MaxSim: each query token takes its maximum similarity over the document's tokens; the maxima sum into the document's score.

Because parts match parts, a query term can align with the exact passage that answers it instead of competing against an averaged summary. The canonical model is ColBERT; ColPali and ColQwen extend the idea to document images, tables, and scans.

What does it cost?

It costs storage and compute: one embedding per token is roughly 10–100× more data than one per document, and MaxSim does far more scoring work than a single dot product. That cost kept late interaction in the reranker seat for years. See is multi-vector retrieval too expensive? for how modern engines changed the math.

When should you use it?

Use it when precision on specific passages decides the outcome: agent retrieval, technical docs, legal and financial search, complex PDFs. For broad topical queries over short texts, single-vector search is cheaper and usually sufficient. The full decision framework is in when to use multi-vector embedding models.

TopK supports multi-vector retrieval natively as a first-stage query option, not a separate system to operate. Its SMVE encoding runs late interaction at roughly 5–8× lower latency than PLAID and MUVERA at competitive recall (March 2026). Enabling it is a single schema annotation:

from topk_sdk.schema import text, semantic_index

client.collections().create(
    "docs",
    schema={"text": text().index(semantic_index())},
)

TopK embeds the text with its late-interaction model and serves MaxSim-scored retrieval from that one line. The multi-vector search guide covers querying and bring-your-own-embeddings setups.