CoFree: Fixing Reasoning Collapse in LLM-based Embedding Learning
_reachsumit · x · 2026-09-18
A new arXiv paper identifies "reasoning collapse" in LLM-based embedding learning: specializing toward embedding objectives either suppresses useful reasoning or produces retrieval-irrelevant text.
The authors propose CoFree, a two-stage framework: reference-guided supervised fine-tuning first restores reasoning ability while preserving representational strength; a second RL stage applies dual rewards (embedding-oriented and reasoning-oriented) to guarantee fine-grained relevance reasoning, turning embedding learning into a reasoning-guided search. CoFree-4B achieves strong average results across benchmarks.
More from Research
- Anil Seth's 'Conscious AI and Biological Naturalism' collection published in BBS with 50 commentaries — anilkseth · 2026-09-18
- NanoGPT Speedrun hits new 68.0s record with 96-dim QK and packed FP8 attention — kellerjordan0 · 2026-09-18
- Solo dev open-sources Laya: 421M non-autoregressive decision model hitting 35ms forward passes — Nandakishor_ml · 2026-09-18
- AI agents helped build scBaseCount: 502M uniformly processed cells, published in Cell — anshulkundaje · 2026-09-18
- Google's ToolGrad flips tool-use dataset generation: answers first, queries later — AxSaucedo · 2026-09-18
- Cognitive subtest profiles are almost never meaningful, psychologist warns — soumitrashukla9 · 2026-09-18