turbovec: Rust vector index fits 10M document vectors in 4GB RAM and outpaces FAISS
tom_doerr · x · 2026-09-10
turbovec is a Rust vector index with Python bindings, built on Google Research's TurboQuant quantization algorithm.
- Memory: a 10M-document corpus that takes 31GB as float32 fits in just 4GB.
- Speed: hand-written SIMD kernels (NEON SDOT/SMMLA on ARM, AVX-512 VNNI and vpermb on x86) beat FAISS IndexPQFastScan in every measured config — averaging 3.4× at 4-bit and 23% at 2-bit.
- Online ingest: no train step or parameter tuning; vectors are indexed as added, with incremental saves via sync(path).
More from Infra
- Keras ships ZeroModels: 100+ model families in pure Keras 3, runnable on any backend — fchollet · 2026-09-10
- Cohere moves to NVIDIA Blackwell, cutting token costs and TTFT by 30–50% — cohere · 2026-09-10
- Author uses local AI models to review book manuscripts for $0 in tokens — walkingriver · 2026-09-10
- Musk's Colossus data center fuels massive local backlash, reports Scientific American — scientificamerican · 2026-09-10
- Analyst: Huawei to undercut US AI stack with cheaper chips tuned for Chinese models — 2C_ornot2C · 2026-09-10
- Photon 2.2 Ships Optimized Local Inference for Ampere Through Blackwell GPUs — JFPuget · 2026-09-10