int8 search + fp32 rescoring lifts recall@100 from 97.24% to 99.94%
tomaarsen · x · 2026-11-10
tomaarsen shares a cheap trick to negate int8 quantization loss in vector image search: use the int8 index to fetch candidates, load their fp16 embeddings from disk, and rescore against the fp32 query to get a new top 30. Versus exact fp32 search, plain int8 gets 97.24% recall@100, while int8 + fp16/fp32 rescoring reaches 99.94% — with minimal added cost before loading the actual images for the user.
More from Infra
- Claude Fable 5.2 and Mythos 5.2 spotted on AWS Bedrock, priced 20% below 5.1 — testingcatalog · 2026-11-10
- e-means paper trick: IVF approximate assignment cuts clustering cost 34x at 10B vectors — antoine_chaffin · 2026-11-10
- Telekom, NVIDIA and SAP expand German AI cloud, boosting capacity 50% — NinaDSchick · 2026-11-10
- Two-person YC-backed Archgen wins chip layout contest against 131 entries — testingcatalog · 2026-11-10
- NVIDIA Dynamo in 5 minutes: OpenAI-compatible frontend enables backend hot-swap and KV cache reuse — NVIDIA Developer · 2026-11-10
- TensorFold lands in Omarchy: dhh hails it as key to local AI recipes on NVIDIA — udmrzn · 2026-11-10