Sentence Transformers maintainer demos int8-quantized retrieval in milliseconds

Tom Aarsen demonstrated a low-cost retrieval pipeline using int8 quantization and Usearch that searches 1.4M embeddings in 2ms, with fp32 rescoring boosting recall@100 from 97.24% to 99.94%.

2026-11-10 ~ 2026-11-10 · 4 related posts

• int8 embeddings plus Usearch index: 1.4M vectors searched in under 2ms on CPU — tomaarsen · 2026-11-10
• int8 quantization plus fp16 rescoring: 1.4M embeddings retrieved in under 2ms — tomaarsen · 2026-11-10
• int8 search + fp32 rescoring lifts recall@100 from 97.24% to 99.94% — tomaarsen · 2026-11-10
• Search 1.4M Images in 15ms: int8 + Rescoring Hits 99.94% Recall with Sentence Transformers — tomaarsen · 2026-11-10