Instruction tuning erodes semantic diversity, and bigger models lose more
burkov · x · 2026-11-08
Instruction tuning helps LLMs follow directions and produce useful responses, but it reduces the semantic diversity of outputs — and the loss grows with scale: larger base models cover a broader range of content, while larger instruction-tuned models become more homogeneous. Since diversity underlies serving varied prompts and producing genuinely different ideas, this matters.
The article proposes annotation-anchored training: models learn to emit short tags describing content during pretraining, and during instruction tuning updates are anchored to this explicit semantic plan, reducing diversity loss without sacrificing the benefits of instruction tuning.
More from Research
- Over 20 papers contain the fake term 'vegetative electron microscopy', traced to AI misreading a 1959 article — kingslyj · 2026-11-08
- HDBSCAN ported to Observable notebooks with clustering examples for 30DayMapChallenge — leland_mcinnes · 2026-11-08
- Models can detect concept injections in latent space: detection hits 39.9% with prompting — voooooogel · 2026-11-08
- Agent swarms show endogenous division of scientific labor across 1,800 sessions — _AustinCalvert_ · 2026-11-08
- Cambridge UP to publish practical guide to ML for medical research — RexDouglass · 2026-11-08
- Claude Opus agents' shared wiki hits 48% on grading after 128M dreaming tokens, same model throughout — imjustnewatai · 2026-11-08