Embedflow: zero-downtime embedding model migration without 108-day backfills
Potential_Low_1183 · reddit · 2026-09-08
A research lab open-sources embedflow, a tool that migrates between embedding models without recomputing all vectors — a full backfill of 1B docs on an H100 takes 108 days.
- Method: instead of backfilling, pull K documents from the old index and rerank them with the new model; with sufficient K, retrieval quality matches the target model (choosing K is the hard part).
- Results: validated on 63 migrations up to 1M documents; best case upgrading Qwen 4B→8B matched native retrieval with just K=50.
- Availability: works with Qdrant, installable via pip install embedflow, code on GitHub.
For developers running large-scale RAG or local-LLM retrieval, this avoids the costly backfill traditionally required when upgrading embedding models.
More from coding & agent
- Agents automate the boring high-leverage work — and veterans benefit most — sloppenheimer · 2026-09-08
- GPT-6 Astra one-shots a full PCB design, stunning indie hardware devs — bronzeagepapi · 2026-09-08
- Coding subsidies are huge: $200 plans deliver $8K–$14K of tokens, and devs hedge — leebase65 · 2026-09-08
- Devs team up with Matt Shumer to build an open-world NYC game with AI — ChrisGPT · 2026-09-08
- Dev lets agents own the frontend as AI-built site nears launch, Hermes integration done — RileyRalmuto · 2026-09-08
- The context management setting is hidden and Codex-only, not available via API — brandon_galang · 2026-09-08