Toby Ord doubles down: pretraining scaling is slowing and has little headroom left
tobyordoxford · x · 2026-09-09
In an exchange with David Manheim, Dwarkesh Patel, and others, Toby Ord reiterates that pretraining scaling is slowing and doesn't have much headroom left. He clarifies that his earlier criticism targeted 'just add compute / just add data' approaches, not the idea that additional human work could still improve the pretraining process itself.
Related event: Scholars Debate Whether Pretraining Scaling Is Slowing Down(4 posts)→
More from AGI Musings
- Cognitive mercantilism: AI panic eerily mirrors early capitalism's anxieties — adityaag · 2026-10-27
- US chases AGI dominance while China, EU and ASEAN treat AI as public infrastructure — BubblyExtent1693 · 2026-10-27
- Lab monkeys at $30-80K each: a 4-year breeding cycle is the real preclinical drug bottleneck — zebird0 · 2026-10-27
- CJR: journalists should stop hyping AI's supposed humanity — just say 'rogue' — ShakeelHashim · 2026-10-27
- PhD applicant emails up 4x YoY due to AI — writing without AI now makes you stand out — AI4Code · 2026-10-27
- Weil's collapse shows BigLaw dies from eroding belief, not a broken business model — jkubicki · 2026-10-27