ZeroEntropy: Matryoshka embeddings aren't lossless — learned projections dominate the whole MRL frontier
MrAshokRaja · x · 2026-10-26
Embedding company ZeroEntropy publishes 'Matryoshka Is Dead', showing via their zembed-1 evals that Matryoshka Representation Learning (MRL) is not lossless: every MRL-trained configuration sits on the same strict Pareto frontier — you can slide along it, not move it — while a learned post-hoc projection dominates the entire MRL frontier at full dim and in head-to-head comparisons.
Why public literature misses this
- Top models cluster within 1–2 points on MTEB; ZeroEntropy's zerank-1 and zerank-1-small (an order of magnitude apart in parameters) differ by <1% on public evals but >10% on internal benchmarks — public benchmarks are insensitive to nuanced frontier regressions.
What they shipped instead
- Six learned projection matrices for zembed-1, on Hugging Face, or via a dimensions parameter in the zembed-1 API.
Discussion thread: random truncation's robustness is surprising; since MRL adds training cost and constraints (pre-set truncation dims), skipping it is simpler and cheaper unless extreme compression is needed. Researcher srchvrs adds that NanoBEIR-scale benchmarks and stale BM25-based QRELs further mask regressions.