Sutton: brute force isn't intelligence; Langford cites 2011 n-gram era to frame scale-invent cycle
weijie444 · x · 2026-11-08
Richard Sutton reiterates his stance: brute force computation is not enough for intelligence, clever new methods are needed, and the science of AI is far from finished.
John Langford responds that the scale-invent-repeat cycle goes much deeper into history than the 'bitter lesson' discussion: people once claimed language modeling was solved by n-gram scaling. He points to their 2011 paper 'A Reliable Effective Terascale Linear Learning System' (with Agarwal, Chapelle, Dudík), which trained linear predictors with trillions of features and billions of examples in an hour on a 1000-machine cluster — the most scalable linear learning system reported at the time.
The exchange adds historical depth to the scaling-vs-methods debate: every scaling era has been declared terminal before new methods emerged.
More from AGI Musings
- Ajeya Cotra's rough modeling suggests new AI risk report points to shorter timelines than AI 2027 — eli_lifland · 2026-11-08
- Acemoglu: AI should complement human skills, not mimic them — math_rachel · 2026-11-08
- JEV Gets Its First Paper as Researchers Split on Research Merit vs Closed Business Model — teortaxesTex · 2026-11-08
- "Humans overfit to confirmation too" — a witty mode collapse analogy for AI slop debate — flowersslop · 2026-11-08
- bayeslord: Stop treating AI as a monolith — four distinct goals demand precise language — sebkrier · 2026-11-08
- AI doesn't change how unique your niche interests are, argues dev pixlpa — pixlpa · 2026-11-08