Hinton: new-language experiment shows LLMs truly understand, not just predict next word

coherence · x · 2026-11-20

Geoffrey Hinton pushes back on the claim that LLMs are "just predicting the next word," arguing it falls apart when models must understand something they've never seen.

He cites an experiment by a former graduate student who defined a new programming language with unusual syntax, gave a model a program in it, and asked what the program would do — the model answered correctly. Hinton says correctly interpreting a novel language definition requires genuine understanding, not mere next-token prediction.

Related event: Hinton Pushes Back on "LLMs Just Predict the Next Word"(2 posts)→

Original post →

More from AGI Musings

AGI Musings channel →