Paper: embeddings are robust to truncation without Matryoshka learning, except heavy truncation
srchvrs · x · 2026-10-26
srchvrs summarizes Takeshita et al.'s EMNLP 2026 Findings paper 'To MRL or not to MRL': (1) the robustness of random truncation is surprising; (2) since MRL adds training cost and constraints (pre-determined truncation dimensions), skipping it is simpler and more cost-efficient unless extreme compression is needed. Across multiple models and tasks, truncated embeddings of non-MRL models match or outperform MRL-trained ones unless reduced by at least 80% — suggesting truncation robustness may not come from MRL. Code is available for reproduction.
Related event: Study Questions Value of Matryoshka Embeddings(3 posts)→
More from Research
- OpenAI's Aaron Lou on diffusion LMs vs autoregressive: why he left a Stanford PhD — aaron_lou · 2026-10-26
- Hamel Husain flags poor benchmark quality: Epoch audits find Terminal-Bench, OSWorld 2.1 all flawed — HamelHusain · 2026-10-26
- RExBench: best coding agent solves only 33% of autonomous research extension tasks — najoungkim · 2026-10-26
- AlphaGeometry solved hard problems by brute force; brevity incentives might help — torchcompiled · 2026-10-26
- All 7,898 NeurIPS 2026 papers now browsable on Papers with Code — NielsRogge · 2026-10-26
- JHU paper uncovers privacy-hallucination tradeoff in differentially private LLMs — danish037 · 2026-10-26