Hornet scales HNSW to a billion vectors by quantizing embeddings to 2 bits
jobergum · x · 2026-11-20
Hornet's CTO Henning Baldersheim shares how they scale HNSW-based retrieval to a billion vectors.
- Goals: true real-time (writes visible immediately), high QPS with low predictable latency, balanced hardware use, and good recall. Design partners run billions of vectors in production with rising QPS from agent workloads.
- Why HNSW: brute force over 1B×1024-dim bfloat16 vectors would read 2TB per query. HNSW's query cost grows far slower than the graph and supports immediate visibility.
- Key techniques: keep few, large graphs (splitting data across many graphs forces every query to traverse all of them); since memory is the bottleneck, vectors are quantized to 2 bits with full-resolution copies on disk for rescoring.
- Open question: whether recall would hold as a graph grew from 100M to 1B vectors — the core thing they set out to test.
Related event: Scaling HNSW to Billions of Vectors(2 posts)→
More from Infra
- DeepMind researcher: global AI compute growing at least 10x per year — 141_1337 · 2026-11-20
- GPU shortage debate: companies with real AI demand are still massively compute starved — matt_slotnick · 2026-11-20
- Microsoft's Bing Web IQ Goes GA on Azure: Auto Orchestration Delivers Fresh Grounding for Agents — dejanseo · 2026-11-20
- exe.dev ships Rex integration, making remote VMs the default agentic dev workstation — davidcrawshaw · 2026-11-20
- Ex-Waymo exec Abe Murray: CUDA and ISAs won't stay moats, huge chip innovation incoming — IgorCarron · 2026-11-20
- MIT and NVIDIA's QDA uses agents to auto-design LLM quantization recipes, beating top public ones by up to 2.2 points — omarsar0 · 2026-11-20