Why Does Filtering Break HNSW?

What goes wrong with post-filtering?

The math is unforgiving: fetch top-100 and apply a filter that matches 1% of the corpus, and the expected number of surviving results is about one. Fetch-more-and-hope (top-1,000, top-10,000) burns latency and still offers no guarantee. That is the "zero results" failure mode, and it appears exactly on the queries where the filter mattered most.

What goes wrong with filtering during traversal?

HNSW's speed comes from greedy hops through a well-connected neighborhood graph. Mask off 99% of nodes and the connected structure the search depends on effectively disappears: paths to matching regions run through non-matching nodes. Engines compensate by visiting more of the graph, which is the timeout failure mode: latency grows as selectivity tightens. Both failure modes, and why they worsen with predicate selectivity, are analyzed in ACORN (Patel et al., SIGMOD 2024).

What does a real fix look like?

The fix is selectivity-adaptive execution inside the engine. Broad filters can stay close to normal graph search; highly selective filters should flip to searching the matching subset directly (at the extreme, brute-forcing a few thousand matching vectors beats wandering a masked graph). Demand one thing from any vendor: published filtered-search latency across selectivities, not just unfiltered numbers.

There's a deeper reason bolted-on filtering can't be fixed: as TopK's engineering team puts it, "the distribution of embeddings and the distribution of metadata are not strongly correlated". A vector index organizes data by semantic neighborhood, so the documents matching your filter are scattered uniformly across it, and no traversal order can find them efficiently. Filtering has to be a first-class operation of the query engine, not a mask over a vector index.

TopK's engine was built to invert the usual curve: its stated design goal is that highly selective queries get faster, not slower, since a tight filter shrinks the candidate set the engine actually scores.

The measurements bear it out at both ends of the scale: ~62ms p99 on 1M documents and ~115ms on 10M across filters selecting 100%, 10%, and 1% of the corpus (March 2025), and at one billion documents (July 2025), dense search improves from ~60ms p99 unfiltered to ~30ms with filters, with no degradation in result quality. Fuller selectivity sweeps are on the benchmarks page.

In practice, filters compose directly with the vector query:

from topk_sdk.query import select, field, fn, match

docs = client.collection("books").query(
    select(
        "title",
        similarity=fn.vector_distance("title_embedding", [0.1, 0.2, ...]),
    )
    .filter(match("catcher"))                  # keyword predicate
    .filter(field("published_year") > 1980)    # metadata predicate
    .sort(field("similarity"), asc=False)
    .limit(10)
)

The query documentation covers the full predicate syntax.