STE regularization compresses ColBERT 5x with 99.4% retrieval retention
antoine_chaffin · x · 2026-11-11
Antoine Chaffin points to a new Hugging Face article building on hierarchical pooling (which already halves ColBERT storage without hurting retrieval): applying the same STE-based regularization used to fix MUVERA/SMVE pushes this to 5x compression with 99.4% retention, without degrading full-token performance. Key insight: the best way to achieve compressability is to train for it directly. The piece covers matching budget to objective, adaptive per-document budgets, and releases LateOn-hpool-regularized and related models. He also notes MTRL's value for multi-vector models and argues embedding dimensions are still too large.
More from Infra
- DeepSeek V4.1 Flash Runs at 20 tokens/s on a Single 3090, 2x Possible with More RAM — casper_hansen_ · 2026-11-12
- Gemini Decision 1 costs $345 per 1M tasks, 3x pricier than leader Drex Max v1.1 — ArtificialAnlys · 2026-11-12
- Cloudflare's weekend-trained hybrid model cuts agent cost from 40K to 1.8K tokens — TheTuringPost · 2026-11-12
- Inference providers buy their own chips, scatter across dozens of less secure small clouds — steph_palazzolo · 2026-11-12
- Microsoft paper details Maia accelerator and software-defined dataflow for trillion-token-scale serving — thoefler · 2026-11-12
- LMSYS megakernel pushes decoding to 1,109 tok/s on 8× GB300, launches cut 1450→97 — ying11231 · 2026-11-12