Turbovec: Rust vector index fits 10M docs in 4GB and beats FAISS by 3.4x at 4-bit
bibryam · x · 2026-09-21
Turbovec is a local Rust vector index for RAG with Python bindings, built on Google Research's TurboQuant algorithm — a data-oblivious quantizer requiring no separate training phase. Its v1.0 stabilizes the v7 on-disk format; the repo has 17.2k stars.
Key points:
- 2-bit/4-bit compression: a 10M-document corpus drops from 31GB RAM (float32) to 4GB
- Online ingest: add vectors and they're indexed immediately — no train step, no rebuilds as the corpus grows
- Hand-written SIMD kernels (NEON SDOT/SMMLA on ARM, AVX-512 VNNI and vpermb on x86, with AVX2/scalar fallbacks) beat FAISS IndexPQFastScan in every measured config — averaging 3.4x faster at 4-bit and 23% at 2-bit
- Incremental, crash-safe saves via sync(path)
More from Infra
- Meta data center floods Louisiana school district with $50,000 teacher bonus checks — victor_explore · 2026-09-21
- Substrate bets the agent loop is the new scheduling loop with fast suspend/resume — rakyll · 2026-09-21
- Agent Substrate open-sourced: agent runtime with 10x container density, sub-500ms resume — rakyll · 2026-09-21
- Huawei's Guo Ping: data centers next need 'book nerds' to route tokens, not coders — teortaxesTex · 2026-09-21
- Chris Albon jokes he'll be 'post-economic' as local compute beats cloud models by 2027 — chrisalbon · 2026-09-21
- GPUs are the commodity; the AI software stack above them will decide the winners — ypatil125 · 2026-09-21