AnswersEmbeddings in Production

How Do You Upgrade Embedding Models?

Q: How do you swap embedding models in production with zero downtime, and how much of that pain disappears when the database owns the embedding pipeline?

The safe pattern is blue-green at the index level: build a second index with the new model, backfill it from source text, dual-write both indexes during the transition, compare quality in shadow, flip reads to the new index, and keep the old one warm until you'd no longer roll back. There is no in-place upgrade, because vectors from different models occupy incompatible spaces, so a half-migrated index is a broken index. The cutover is the easy part. The real work is the re-embedding pipeline: you own it end to end, and query-time and ingest-time embeddings must stay in lockstep.

How do you swap embedding models in production with zero downtime, and how much of that pain disappears when the database owns the embedding pipeline?

2 min readJuly 2026

Why can't you upgrade in place?

A query embedded with the new model scores garbage against documents embedded with the old one. Any state where one index holds both is serving wrong results. That single fact forces the parallel-index shape of every safe migration.

What's the sequence?

Backfill: re-embed the corpus with the new model into a new index (dedupe first: don't pay twice for redundancy).
Dual-write: new and updated documents flow to both indexes so neither goes stale.
Shadow-test: run real traffic against both; compare recall and answer quality offline before any user sees a change.
Flip reads, then watch: cut over behind a flag; the old index is your instant rollback.

Blue-green at the index level: no moment ever serves mixed vector spaces.

Backfill index B

re-embed from source

Dual-write A + B

Shadow-test

Flip reads to B

Keep A for rollback

Where does it actually go wrong?

Embedder skew causes the classic production incident: one version of the model (or one preprocessing path) embeds the queries while another embedded the documents. Nothing errors; results are just quietly worse. Any migration plan is mostly a plan for keeping the ingest-side and query-side embedder identical at every moment.

What changes when the database owns embedding?

Most of the pipeline disappears. When the engine embeds at both ingest and query time (managed inference), skew becomes structurally impossible (one embedder, one place), backfill becomes a database-side operation instead of an external batch job you build, and the migration collapses to: create index with model B, backfill, flip.

TopK's semantic_index is the concrete version: one schema annotation, no external embedding pipeline, with embedding inference running inside the system at over 1.5B tokens per hour, which makes a model upgrade a schema change rather than an infrastructure project.