Vector Search & Retrieval Answers
Quick, definite answers to the specific questions that come up when you're building retrieval for AI applications: multi-vector search, hybrid ranking, retrieval for agents. Each page answers one question up front, states the tradeoffs honestly, and links the primary sources.
Multi-Vector Retrieval
What Is Multi-Vector Retrieval?Multi-vector retrieval stores one embedding per token and scores documents with MaxSim, so precise queries match exact passages instead of a lossy document summary.What Is MaxSim?MaxSim sums each query token's best match against a document's tokens. It is the scoring operator behind multi-vector (late-interaction) retrieval.When to Use Multi-Vector Embedding ModelsUse multi-vector models when queries target specific details in long or complex documents; use single-vector when queries are broad or cost dominates.Is Multi-Vector Retrieval Too Expensive?It was: token-level embeddings cost 10–100× more storage. Two-stage designs and MaxSim approximations like SMVE now make it a viable first stage.
Hybrid Search & Ranking
When Do You Need Hybrid Search?Use hybrid search when queries mix natural language with exact tokens like names and codes. Vectors blur exact terms; keywords miss paraphrases.RRF vs Score Fusion vs True Hybrid: Which Should You Use?RRF merges lists by position, score fusion tunes weighted scores, and true hybrid scores every signal in one query. Start with RRF; move up as you measure.How Should You Weight Hybrid Scores?There is no single right weight: short queries lean lexical, verbose ones lean dense. Condition weights on the query, or use RRF and skip them.Do You Still Need a Reranker?Add a reranker when the right documents reach your top 100 but not your top 5. With a multi-vector first stage, measure first: it may be redundant.Can You Run Vector Search in SQL?Yes, two ways: extensions like pgvector add distance operators to Postgres, and TopK speaks the Postgres protocol so hybrid search is ordinary SQL.
Agents & RAG
Why Does RAG Retrieve the Wrong Chunks?Wrong chunks come from four places: chunk boundaries, lossy embeddings, missed exact tokens, and filters that empty results. Your failed queries tell you which.Why Does RAG Fail for Agents?Agents fire many precise, long-tail queries: the worst case for single-vector RAG, which summarizes away the details. Fix retrieval, not the model.Why Does RAG Fail on Financial Tables?Single-vector embeddings compress a whole page into one summary, so the row you need averages away into noise. Token-level retrieval matches at the cell.How Do You Audit an AI Answer?An answer is auditable when every claim cites a document and page a human can open. That requires citations from retrieval, not citations written by the model.
Scale & Architecture
Why Object Storage for Vector Search?At billions of vectors the dominant cost is keeping data hot. Object storage decouples storage from compute, so cost tracks query load, not data size.Why Do Upserts Slow HNSW Reads?Reads and writes contend on one graph: inserts rewire lists mid-traversal and deletes leave tombstones. Sustained upserts tax every read.Why Do Vector Databases Re-Shard?Stateful engines pin data to nodes, so growth forces risky migrations. Storage-compute-separated engines pin nothing; there is nothing to rebalance.Does Quantization Hurt Accuracy?Yes, but recoverably: quantize for candidate selection, then rescore the shortlist at full precision. The loss lands in the tail, not the top results.How Do You Chunk Redundant Documents?Deduplicate before you embed: canonicalize versions, hash-dedupe chunks, and reach history through filters. Indexing every version multiplies cost.How Do You Benchmark Vector Search?Benchmark your own workload: mixed read/write load, your filters, and latency percentiles at a fixed recall target. Read-only QPS flatters every engine.Which Recall Number Actually Matters?Index recall measures whether ANN approximates exhaustive search. Retrieval quality measures whether the right document surfaces. Evaluate quality first.
Filtering & Multi-Tenancy
Why Does Filtering Break HNSW?The graph was built without knowing your filter. Post-filtering empties results; in-traversal filtering breaks connectivity. Native filtering fixes both.How Do You Enforce RBAC in Vector Search?Store access metadata on every document and enforce entitlements as a query-time filter in the engine. Post-filtering leaks; per-role indexes explode.Collections or Filters for Multi-Tenancy?Default to one shared collection with a mandatory tenant filter; split out only huge or contractually isolated tenants. Idle-collection cost decides.
Embeddings in Production
Can You Convert Vectors Between Models?No. Different models produce incompatible spaces, and learned mappings lose the distinctions you upgraded for. Plan to re-embed, and make it cheap.How Do You Upgrade Embedding Models?Blue-green the index: backfill with the new model, dual-write, shadow-test, flip reads, keep the old index for rollback. Model vectors never mix.