Hinton Pushes Back on "LLMs Just Predict the Next Word"
Geoffrey Hinton argues that calling LLMs "next-word predictors" fails when models handle genuinely novel inputs, citing a new programming language a model could understand. Since LLMs predict tokens, they can in principle understand any tokenizable structure, from language to video.
2026-11-19 ~ 2026-11-20 · 2 related posts
- Hinton: the 'just predicting next word' argument falls apart — LLMs can read a brand-new programming language — CatAstro_Piyush · 2026-11-19
- Hinton: new-language experiment shows LLMs truly understand, not just predict next word — coherence · 2026-11-20