IR Papers Weekly Vol.173: Spectral Compression for ColBERT, MoE Multimodal Embeddings from Meta
_reachsumit · x · 2026-09-13
Sumit published Vol.173 of his weekly information retrieval papers digest, covering 10 papers from Microsoft, Meta, ByteDance, Alibaba and academic groups:
- EigenLI (Microsoft): exploits the observation that document token embeddings live in a low-dimensional subspace, representing each doc with top-k eigenvectors of its second-moment matrix to cut storage/compute for late-interaction models like ColBERT
- In-place embedding corrections for feedback-driven dense retrieval (Amendola et al.)
- Meta: scaling multimodal embeddings via Mixture-of-Experts; plus an agent harness for long-horizon experimentation on industry-scale recommenders
- ByteDance: ultra-long sequence modeling in recommendation with low-rank caching
- Alibaba: offline LLM reasoning for online recommendation
- Also: end-to-end training of search agents, adaptive tree expansion for large-scale RAG evidence synthesis, generative late-interaction embeddings for visual document retrieval, and distillation-based soft context compression for RAG
More from Research
- CTC-TTS paper replaces MFA with CTC alignment for low-latency dual-streaming TTS — kastnerkyle · 2026-09-13
- Full-duplex voice models still fall short on humanlike zero-overlap turn-taking — alexisgallagher · 2026-09-13
- Yoav Goldberg: new robot models aren't zero-shot — they're trained to the max on each task — benno_krojer · 2026-09-13
- Synthetic Persona Pretraining paper aligns LLMs from token zero, boosting jailbreak robustness — _arohan_ · 2026-09-13
- Turn prediction benchmark: best systems have poor recall and high false positives — alexisgallagher · 2026-09-13
- Math and AI publishes declaration on AI's impact on human intellectual activity — yoavartzi · 2026-09-13