Counter to Sutton: AI progress is bottlenecked by tedious scaling work, not clever new architectures
burny_tech · x · 2026-11-08
Responding to Richard Sutton's claim that brute-force computation is insufficient and clever new methods are needed, @sensho argues the working scaling law is real but bottlenecked by top talent's willingness to do unglamorous work: pre-training pipelines, mid-training, RL, mech interp. Coding agents improved via great environments + new RL algorithms; DeepSeek's gains came from pre-training pipelines and data quality — not a revolutionary new architecture.
More from AGI Musings
- Humanity's Greatest Achievement May Be an Intelligence That Surpasses Us — VraserX · 2026-11-10
- 430,000 AI Micro-Dramas in 8 Months: A Visitor's Take on China's AI Content Boom — JacquesThibs · 2026-11-10
- FOCS 2026 Grad Student Feedback: US Students More Optimistic, Many Feel No Agency — thegautamkamath · 2026-11-10
- Reddit essay: most of what people hate about AI comes from the humans steering it — teleport66 · 2026-11-10
- Underrated question: what do the diverse forms of recursive self-improvement look like? — repligate · 2026-11-10
- 'Bitter lesson for robotics' gets a sharp rebuttal: it's still human-orchestrated end to end — burny_tech · 2026-11-10