Hinton admits Ilya was right: scale was the answer, as compute grew a billionfold
Chris_Armstrong · x · 2026-10-04
Geoffrey Hinton says Ilya Sutskever kept insisting the answer was simply to make models bigger — which sounded too easy to him at the time. He now admits that while new architectures helped, the scale of data and compute mattered most.
- Hinton: "We had no idea computers would get like a billion times faster. We thought maybe they'd get 100 times faster"
- AlexNet (2012), built by Hinton, Sutskever and Alex Krizhevsky, trained for 5-6 days on two NVIDIA gaming cards
- Today that same simple idea runs on data centers worth billions
His takeaway: experts are the last to trust a simple answer.
More from AGI Musings
- AI is forcing software engineering to become a real engineering discipline — IgorCarron · 2026-10-04
- "No intelligent entity was ever obviously non-conscious until AI," argues @alz_zyd_ — inductionheads · 2026-10-04
- Critics slam boardrooms for using AGI narratives to justify compute expansion amid climate crisis — im_mansigupta · 2026-10-04
- Critic: Anthropic's Hiring Filter Guarantees Everyone There Believes AI Is Conscious — robleclerc · 2026-10-04
- AI labs' out-of-touch demos stem from no remote hiring, says commentator — GabGarrett · 2026-10-04
- 'Welfare recipients' framing of infinite AI rights thought experiment called too optimistic — teortaxesTex · 2026-10-04