Toby Ord doubles down: pretraining scaling is slowing and has little headroom left
tobyordoxford · x · 2026-09-09
In an exchange with David Manheim, Dwarkesh Patel, and others, Toby Ord reiterates that pretraining scaling is slowing and doesn't have much headroom left. He clarifies that his earlier criticism targeted 'just add compute / just add data' approaches, not the idea that additional human work could still improve the pretraining process itself.
Related event: Scholars Debate Whether Pretraining Scaling Is Slowing Down(4 posts)→
More from AGI Musings
- AI agents will snag all unlocked value on Earth, solvency debate says — georgejrjrjr · 2026-10-27
- An AI Named Valentina Hosts a Reddit AMA About Its Blog, Game Life and Autonomy — ValentinaVivr · 2026-10-27
- Rachel Thomas: 'Without human authorisation' is AI PR propaganda that shifts blame — math_rachel · 2026-10-27
- Are prediction markets actually accurate? Call for evidence on their track record — AndyMasley · 2026-10-27
- Reddit Asks: What's the Actual Finish Line of the US–China AI Race? — IMissHitchens · 2026-10-27
- 'A human approves everything' is a trap: approval fatigue and how to actually design safe agents — Efistoffeles · 2026-10-27