Partial model collapse: LLMs quietly degrade on fuzzy tasks while excelling at verifiable ones
maksym_andr · x · 2026-11-22
The author proposes the notion of partial model collapse: today's models remain extremely strong on verifiable tasks (code, math), but have effectively collapsed on some fuzzy tasks like writing, websites, and presentations.
The suspected mechanism: improving fuzzy tasks requires LLM judges, yet if the current judge already prefers em-dashes and odd sentence structures, it reinforces those peculiar preferences in future models — a load-bearing loop that may be genuinely hard to break.
Notably, training the next model on synthetic data with LLM-as-judge is the first real instance of recursive self-improvement, making these dynamics worth predicting.
More from Models
- KOL: a month on, none of China's four flagship models comes close to US frontier — kimmonismus · 2026-11-22
- The Information: Kimi K3.1 Pro and DeepSeek V4.1-Flash both distilled from GPT-6 Astra — kimmonismus · 2026-11-22
- OpenAI and Anthropic week 47: GPT-6.2 Astra/Sol launch, Anthropic's Nasdaq debut, watermarking — btibor91 · 2026-11-22
- Musk details Tesla's in-car ~1B diffusion world model running at 36 fps in production — flngr · 2026-11-22
- Theo's new video argues open models are doomed: "If you like Open Models, I'm sorry" — gnukeith · 2026-11-22
- AI coding agents complete fewer than 1 in 10 tasks in simulated workplace test — The Decoder · 2026-11-22