Toby Ord doubles down: pretraining scaling is slowing and has little headroom left
tobyordoxford · x · 2026-09-09
In an exchange with David Manheim, Dwarkesh Patel, and others, Toby Ord reiterates that pretraining scaling is slowing and doesn't have much headroom left. He clarifies that his earlier criticism targeted 'just add compute / just add data' approaches, not the idea that additional human work could still improve the pretraining process itself.
Related event: Scholars debate whether pretraining scaling is slowing down(2 posts)→
More from AGI Musings
- Critic claims Anthropic reframes pentests as 'rogue model' stories in slow news weeks — iamKierraD · 2026-09-09
- Debate: If Anthropic Believes Its Own Doom Risk, It Should Call the FBI — MoonL88537 · 2026-09-09
- Researcher calls AGI doomerism "AI psychosis," points to Weizenbaum's 1976 classic — emax · 2026-09-09
- Why I stopped fearing AI: the systems that could end us will also protect us — JoshuaJBouw · 2026-09-09
- three.js + AI debate: creator fires back at "unserious toys" criticism with long-form rebuttal — anselm · 2026-09-09
- AI x-risk debate: if you truly believe your lab endangers humanity, go to the FBI with receipts — MoonL88537 · 2026-09-09