Doing the math: int8 dense embeddings cost ~2x the storage of binary ColBERT
antoine_chaffin · x · 2026-08-25
Pushing back on "multi-vector indexes are too big", antoinechaffin computes storage for MS MARCO (8.8M docs) with a counterintuitive result:
- ColBERT (32 tokens/doc, dproj=128, binary doc-side): 4.5 GB;
- Dense (one 768-dim vector/doc): FP32 27 GB, BF16 14 GB, int8 7 GB—nearly twice binary ColBERT even with int8.
Combining Mixedbread's asymmetric binary quantization results, he argues a standard PLAID index is already smaller than—or comparable to—the >4000-dim single-vector representations most people use today; the problem is diffusion of good infra, not a technical challenge.
Related event: Developers Debunk Late Interaction Misconceptions: Naming and Index Size(9 posts)→
More from Research
- Sliding puzzle video explains why AI reasoning needs guided search, not just sampling — CatAstro_Piyush · 2026-08-25
- Anthropic and AWS sponsor hackathon to decode rare disease using open genome data — CatAstro_Piyush · 2026-08-25
- Contravariance theory for RSA: removing nuisance components restores full contravariance — dyamins · 2026-08-25
- Papers with Code Updates Healthcare AI Page with Benchmarks and Papers — NielsRogge · 2026-08-25
- $40 experiment: Opus 5 hits ~0.94 F1 on ExtractBench, Qwen3.8 matches at 1/3 the price — Ok-Challenge-7810 · 2026-08-25
- EMNLP 2026 registration opens; author deadline Sept 11, Budapest in October — delliott · 2026-08-25