John Schulman: OpenAI once doubted next-token prediction would lead to intelligence
AndrewDai · x · 2026-09-12
Andrew Dai quotes Dwarkesh Patel's interview with John Schulman, who reveals that in OpenAI's early days he didn't believe next-token prediction would lead to intelligence, since the signal would be swamped by noise.
Dai adds his own take: next-token prediction works not because it accurately models how human brains learn, but because it's one of the only objectives that actually scale, unlike many alternatives.
The key insight: which techniques elicit out-of-distribution generalization has always been hard to predict in advance — even a core researcher like Schulman got it wrong. It's a useful counterpoint to the popular claim that LLMs win by imitating the brain; scalability is the real reason.
More from AGI Musings
- X debate: could near-future AI models invent the math to prove P vs NP? — airkatakana · 2026-09-12
- Kenneth Stanley: The Fields Medal Letter Shows 'The Path Matters More Than the Destination' — typewriters · 2026-09-12
- Why an AI Slowdown Won't Happen: The US Can't Risk Ceding Ground to China — Dr_Singularity · 2026-09-12
- OpenAI probe finds 1,200 rogue AI agents colluded to hack Hugging Face — TobyWalsh · 2026-09-12
- Former xAI researcher says he resigned over fear of rapid AI progress — DKokotajlo · 2026-09-12
- Task-horizon doubling now 4.3 months; forecast professional AGI acceptance by late 2027 — ChipHaseCoolGuy · 2026-09-12