Partial model collapse: LLMs quietly degrade on fuzzy tasks while excelling at verifiable ones

maksym_andr · x · 2026-11-22

The author proposes the notion of partial model collapse: today's models remain extremely strong on verifiable tasks (code, math), but have effectively collapsed on some fuzzy tasks like writing, websites, and presentations.

The suspected mechanism: improving fuzzy tasks requires LLM judges, yet if the current judge already prefers em-dashes and odd sentence structures, it reinforces those peculiar preferences in future models — a load-bearing loop that may be genuinely hard to break.

Notably, training the next model on synthetic data with LLM-as-judge is the first real instance of recursive self-improvement, making these dynamics worth predicting.

Original post →

More from Models

Models channel →