Can You Do Multi-Vector Search Through the Elasticsearch API?

What does Elasticsearch offer for multi-vector?

Elasticsearch stores a variable number of vectors per document in a rank_vectors field, and it scores them with a script_score query that calls maxSimDotProduct on the query matrix (Elastic rank_vectors reference). The reference describes the field as a way to rerank documents with MaxSim, and the field takes element_type and dims and nothing else. There is no index or similarity parameter, so the engine cannot search the field the way it searches a dense_vector.

The script_score query scores whatever its inner query returns. Under match_all, which is the example in the reference, the script computes MaxSim for every document in the index, one at a time and without an index to narrow the work. Under a match or a single-vector knn, the script rescores that shortlist, and the multi-vector signal only reaches documents the first retriever already found.

Why is rescoring different from retrieval?

Rescoring can only reorder the shortlist, so a document that a single vector or BM25 misses never gets a MaxSim score, and the brute-force alternative reaches every document by scanning every document. Multi-vector retrieval exists because one pooled vector loses the detail inside a passage that precise queries need, and the documents it loses are the ones that only look relevant token by token. A pipeline that retrieves with the pooled vector and rescores with MaxSim inherits the miss, and the rescoring step cannot recover it.

The script also costs the full MaxSim per candidate, which is the query token count times the document token count in dot products, for every document in the shortlist. Widening the shortlist to catch more misses raises that cost in proportion, and it still never reaches the documents outside it.

What does an indexed multi-vector field change?

An index lets MaxSim choose the candidates instead of reordering someone else's, so a document that only looks relevant token by token can still be found. TopK's multi-vector index, SMVE, does that selection and then scores the shortlist with exact MaxSim. In TopK's measurement (March 2026, ColBERTv2 on BEIR), that gave roughly 5 to 8 times lower end-to-end latency than PLAID and MUVERA at competitive recall, and on MS MARCO, with 8.8M documents, SMVE averaged 39.9 ms for k=100 against 221 to 318 ms for PLAID and 310 to 444 ms for MUVERA.

The production system built on it, semantic_index, measured 52.88% nDCG@10 across all 15 BEIR datasets, within about 1% of exhaustive MaxSim, at 295 QPS and about 75 ms p99, while ingesting over 1.5B tokens per hour (June 2026). Is multi-vector retrieval too expensive covers the cost side of the same measurements.

What does this look like through the Elasticsearch API?

You map the field as rank_vectors, index a matrix per document, and search with a knn clause whose query_vector is a matrix. On TopK, that field is served by a multi-vector index scored by MaxSim, and the search ranks every document in the index by MaxSim rather than rescoring a shortlist.

from elasticsearch import Elasticsearch

es = Elasticsearch("https://<your-topk-es-endpoint>", api_key="<topk-api-key>")

es.indices.create(index="passages", mappings={"properties": {
    "token_embeddings": {"type": "rank_vectors", "dims": 128},
}})

es.index(index="passages", id="p1", document={
    "token_embeddings": passage_token_vectors,  # one vector per token
})

res = es.search(index="passages", knn={
    "field": "token_embeddings",
    "query_vector": query_token_vectors,        # a matrix, one vector per query token
    "k": 10,
})

TopK checks the shapes both ways. A flat vector against a rank_vectors field returns a 400, and a matrix against a dense_vector field returns a 400, so a mismatch fails at the request rather than producing a wrong ranking. The compatibility overview covers the rest of the API, and when to use multi-vector embedding models covers whether your workload needs the field at all.