Microsoft's EigenLI Compresses ColBERT Representations, Beats MUVERA Baseline
_reachsumit · x · 2026-09-09
Microsoft researchers present EigenLI, a training-free spectral method that compresses ColBERT-style late-interaction representations. Key insight: document token embeddings concentrate in a low-dimensional subspace that preserves most retrieval signal, so each document's dominant eigendirections can build a reduced interaction representation. k-EigenLI with k≤32 outperforms k-means and Ward clustering pooling on ColBERTv2 and AnswerAI-ColBERT-small, though GTE-ModernColBERT favors clustering at k=32. The same construction also yields EigenLI-SV, an ANN-compatible single-vector representation that consistently beats MUVERA-like surrogates across datasets and models.
Related event: Microsoft's EigenLI Compresses ColBERT Representations Without Training(2 posts)→
More from Research
- New review: how prefrontal cortex dynamics turn world models into flexible behavior — AnnaCiaunica · 2026-10-27
- LIBERO-MAX: Robot Policies Drop 11–26 Points When the World Changes Mid-Task — drmapavone · 2026-10-27
- Kolab Benchmark Uses Pro Screen Recordings to Show AI Agents Overestimate Collaboration Quality — minnesotanlp · 2026-10-27
- One BBQ Example Pushes LLM Fairness Scores Past 90%, EMNLP Paper Questions Benchmarks — radamihalcea · 2026-10-27
- Real2Sem2Real turns one phone video into 2,000+ robot demos, matching 150 teleop demos — Scobleizer · 2026-10-27
- Understanding MuonH (Hyperball) through weight norm geometry of scale-invariant networks — jasondeanlee · 2026-10-27