ANSWERS
Vector Search &
Retrieval Answers.
Quick, definite answers to the specific questions that come up when you're building retrieval for AI applications: multi-vector search, hybrid ranking, retrieval for agents. Each page answers one question up front, states the tradeoffs honestly, and links the primary sources.
Multi-Vector Retrieval
5 ANSWERSWhat Is Multi-Vector Retrieval?
Multi-vector retrieval stores one embedding per token and scores documents with MaxSim, so precise queries match exact passages instead of a lossy document summary.
What Is MaxSim?
MaxSim sums each query token's best match against a document's tokens. It is the scoring operator behind multi-vector (late-interaction) retrieval.
When to Use Multi-Vector Embedding Models
Use multi-vector models when queries target specific details in long or complex documents; use single-vector when queries are broad or cost dominates.
Is Multi-Vector Retrieval Too Expensive?
It was: token-level embeddings cost 10–100× more storage. Two-stage designs and MaxSim approximations like SMVE now make it a viable first stage.
Can You Do Multi-Vector Search Through the Elasticsearch API?
In Elasticsearch, rank_vectors only rescores candidates through a script. TopK indexes the field for MaxSim retrieval, searched by a knn clause with a matrix.
Hybrid Search & Ranking
6 ANSWERSWhen Do You Need Hybrid Search?
Use hybrid search when queries mix natural language with exact tokens like names and codes. Vectors blur exact terms; keywords miss paraphrases.
RRF vs Score Fusion vs True Hybrid: Which Should You Use?
RRF merges lists by position, score fusion tunes weighted scores, and true hybrid scores every signal in one query. Start with RRF; move up as you measure.
How Should You Weight Hybrid Scores?
There is no single right weight: short queries lean lexical, verbose ones lean dense. Condition weights on the query, or use RRF and skip them.
Do You Still Need a Reranker?
Add a reranker when the right documents reach your top 100 but not your top 5. With a multi-vector first stage, measure first: it may be redundant.
Can You Run Vector Search in SQL?
Yes, two ways: extensions like pgvector add distance operators to Postgres, and TopK speaks the Postgres protocol so hybrid search is ordinary SQL.
Can You Do Hybrid Search Through the Elasticsearch API?
Yes. Send query and knn in one request and the scores add with per-clause boosts, or use rank.rrf to fuse by rank. semantic_text covers having no embeddings.
Agents & RAG
5 ANSWERSWhy Does RAG Retrieve the Wrong Chunks?
Wrong chunks come from four places: chunk boundaries, lossy embeddings, missed exact tokens, and filters that empty results. Your failed queries tell you which.
Why Does RAG Fail for Agents?
Agents fire many precise, long-tail queries: the worst case for single-vector RAG, which summarizes away the details. Fix retrieval, not the model.
Why Does RAG Fail on Financial Tables?
Single-vector embeddings compress a whole page into one summary, so the row you need averages away into noise. Token-level retrieval matches at the cell.
How Do You Audit an AI Answer?
An answer is auditable when every claim cites a document and page a human can open. That requires citations from retrieval, not citations written by the model.
Why Do AI Agents Search With Regex?
A phrase is already a valid regex; case-folding and alternatives are small edits to it. One tool covers every query, so agents use it. The engine must be cheap.
Scale & Architecture
11 ANSWERSWhy Object Storage for Vector Search?
At billions of vectors the dominant cost is keeping data hot. Object storage decouples storage from compute, so cost tracks query load, not data size.
Why Do Upserts Slow HNSW Reads?
Reads and writes contend on one graph: inserts rewire lists mid-traversal and deletes leave tombstones. Sustained upserts tax every read.
Why Do Vector Databases Re-Shard?
Stateful engines pin data to nodes, so growth forces risky migrations. Storage-compute-separated engines pin nothing; there is nothing to rebalance.
Does Quantization Hurt Accuracy?
Yes, but recoverably: quantize for candidate selection, then rescore the shortlist at full precision. The loss lands in the tail, not the top results.
How Do You Chunk Redundant Documents?
Deduplicate before you embed: canonicalize versions, hash-dedupe chunks, and reach history through filters. Indexing every version multiplies cost.
How Do You Benchmark Vector Search?
Benchmark your own workload: mixed read/write load, your filters, and latency percentiles at a fixed recall target. Read-only QPS flatters every engine.
Which Recall Number Actually Matters?
Index recall measures whether ANN approximates exhaustive search. Retrieval quality measures whether the right document surfaces. Evaluate quality first.
Why Is Regex Search Slow at Scale?
Regex is fast per string and slow per corpus, because every document gets read and tested. An n-gram index that finds candidates first removes most of the work.
How Do You Index Text for Regex Search?
Split text into n-grams, keep a posting list per gram, and compile the regex into a boolean expression over the grams it needs. Sparse grams keep it selective.
Can You Replace Elasticsearch Without Rewriting Queries?
Yes, if the engine speaks the Elasticsearch API well enough for the official clients: the product header, your mappings, bulk and search, and the query DSL.
How Do You Run kNN Search Through the Elasticsearch API?
Map a dense_vector field with dims and similarity, send a knn clause with field, query_vector, and k, and put the filter inside knn so it runs before ranking.
Filtering & Multi-Tenancy
3 ANSWERSWhy Does Filtering Break HNSW?
The graph was built without knowing your filter. Post-filtering empties results; in-traversal filtering breaks connectivity. Native filtering fixes both.
How Do You Enforce RBAC in Vector Search?
Store access metadata on every document and enforce entitlements as a query-time filter in the engine. Post-filtering leaks; per-role indexes explode.
Collections or Filters for Multi-Tenancy?
Default to one shared collection with a mandatory tenant filter; split out only huge or contractually isolated tenants. Idle-collection cost decides.
Embeddings in Production
2 ANSWERSCan You Convert Vectors Between Models?
No. Different models produce incompatible spaces, and learned mappings lose the distinctions you upgraded for. Plan to re-embed, and make it cheap.
How Do You Upgrade Embedding Models?
Blue-green the index: backfill with the new model, dual-write, shadow-test, flip reads, keep the old index for rollback. Model vectors never mix.