Interpretability explains the paradox: LLMs plan ahead internally despite next-token prediction
mlpowered · x · 2026-10-19
The author tackles an apparent contradiction: how can models solve hard math problems and write coherent essays if they only predict one word at a time, when humans clearly need to plan ahead for such tasks?
Insights from looking inside models:
- Researchers find parts of the model that specialize in representing a plethora of concepts, from simple colors to nuanced emotions.
- Models use these internal representations to better predict what comes next — e.g., to predict what a character in a play will say, the model learns to infer that character's emotions.
In short, models do plan ahead internally, which resolves the tension between next-token prediction and complex task performance.
Related event: Study Finds LLMs Build Internal Concept Representations and Infer Emotions(2 posts)→
More from Research
- Burkov's Hundred-Page Language Models Book details three architectures with PyTorch code — burkov · 2026-10-19
- FOMO-DEX from Amazon FAR, CMU and Berkeley uses force feedback to fix only contact commands — HaozhiQ · 2026-10-19
- Preprint finds virus-specific T cells persistently activated in 40% of Long COVID cases — alvelda · 2026-10-19
- New evidence suggests consciousness is a dynamic flow, not a static thing — AnnaCiaunica · 2026-10-19
- MEAV: editing LLM alignment at inference time with composable alignment vectors — burkov · 2026-10-19
- Biomedical researcher hails GPT-5 science acceleration paper with OpenAI as his most consequential work — DeryaTR_ · 2026-10-19