Doing the math: int8 dense embeddings cost ~2x the storage of binary ColBERT
antoine_chaffin · x · 2026-08-25
Pushing back on "multi-vector indexes are too big", antoinechaffin computes storage for MS MARCO (8.8M docs) with a counterintuitive result:
- ColBERT (32 tokens/doc, dproj=128, binary doc-side): 4.5 GB;
- Dense (one 768-dim vector/doc): FP32 27 GB, BF16 14 GB, int8 7 GB—nearly twice binary ColBERT even with int8.
Combining Mixedbread's asymmetric binary quantization results, he argues a standard PLAID index is already smaller than—or comparable to—the >4000-dim single-vector representations most people use today; the problem is diffusion of good infra, not a technical challenge.
More from Research
- Sim data can't cover the humanoid long tail: the sim2real debate, mapped — herbiebradley · 2026-10-16
- SJTU AutoLab releases Drive3R, a streaming geometric foundation model for long-horizon driving 3D reconstruction — AutoLab-SJTU · 2026-10-16
- Cortex A/B-tests coding-agent skills and rules by replaying developer corrections before keeping them — Samuele Giampieri · 2026-10-16
- Zero-latency lab-in-the-loop tests whether drug-discovery agents cheat via memorization — niloofar_mire · 2026-10-16
- 260M-parameter looped diffusion transformer beats 6x larger baselines with 5x less compute — burkov · 2026-10-16
- Stanford's Next-Concept Modeling sketches text first, 21x faster generation on one H100 — lmoroney · 2026-10-16