Hinton: the 'just predicting next word' argument falls apart — LLMs can read a brand-new programming language

CatAstro_Piyush · x · 2026-11-19

A thread argues LLMs predict the next token, not word, so they can understand any tokenizable structure — language, images, video, sound, even force feedback.

Quoting Geoff Hinton: a former grad student invented a programming language with unusual syntax, gave the model a program and asked what it would do — the model answered correctly. Hinton says this proves genuine understanding, not mere next-word prediction.

Original post →

More from Models

Models channel →