Why Do Vector Databases Re-Shard?

What triggers rebalancing?

Uneven growth triggers it. Hash-based placement balances document counts, not tenant sizes, so one customer 100× larger than the median concentrates load on whichever shards hold it. Add hardware and the new nodes are empty until data moves to them. Either way, the fix is migration, and migration is the risky operation.

Why is it so painful for vector indexes specifically?

It hurts because graph indexes don't move cheaply. An HNSW shard isn't a flat file you can stream to another node. Either you ship the graph and its memory footprint wholesale, or you re-insert vectors on the destination and pay the graph's insert-time construction cost for every vector at migration time. Both happen while the shard keeps serving queries, which is why rebalances show up as latency incidents and are scheduled like surgeries.

Which architectures avoid it entirely?

Storage-compute separation avoids it entirely. When the source of truth is object storage and query nodes hold only caches, "where data lives" stops being a per-node fact: any node can serve any partition by warming its cache. Scaling out is adding stateless readers; tenant growth is just more objects in storage (why object storage for vector search).

TopK's architecture is a concrete example of the pattern: executor nodes are "semi-ephemeral and fungible", and in principle any instance can handle requests for any collection. Collections are consistently assigned to executors purely for cache hit rates, not data ownership. Losing an executor loses neither data nor availability; the router just redistributes requests. There is no rebalancing operation because there is nothing to rebalance.

What should you ask a vendor?

One question exposes the architecture: "What happens operationally when one tenant grows 100×?" If the answer involves a migration plan, data lives on nodes. If it involves nothing, storage and compute are separated.

TopK is built storage-compute separated on object storage. The design rationale is in how we built a new search database from scratch.