Blanche Minerva's five-step pipeline shows why "just next-token prediction" no longer fits LLMs
BlancheMinerva · x · 2026-09-27
Pushing back on the "LLMs are just stochastic parrots / next-token predictors" framing, Blanche Minerva sketches a pipeline: take a transformer and (1) pretrain it on text, (2) teach special tokens for tool calls, (3) teach command line use, (4) teach CoT for longer reasoning and planning, (5) teach multi-turn conversation.
She argues that even "pure" LLMs are no longer meaningfully captured by that model — largely true in 2021, not anymore. Mitchell responds that he and Emily Bender have written the opposite piece, explaining how the phrase refers to LLMs.
Related event: Melanie Mitchell Sparks Debate: Today's Systems Are Not LLMs(31 posts)→
More from AGI Musings
- UK's new AI Economics Institute hiring up to 15 research economists — TaniaBabina · 2026-11-11
- Not Boring: the three arguments that AI isn't a bubble are exactly how bubbles form — Not Boring (Packy McCormick) · 2026-11-11
- Viral post: five years of 'maybe next year,' then 2026 arrived — hey_abusiddik · 2026-11-11
- Jeremy Howard: LLMs "cosplay understanding," like the man in Searle's Chinese room — Machine Learning Street Talk · 2026-11-11
- 'Chat with your data' misses the point — people want a smart coworker — vboykis · 2026-11-11
- Ben Todd: AI task time horizons appear superexponential in AECI, not just revenue — ben_j_todd · 2026-11-11