Oxford's Toby Ord estimates pre-training compute returns near-linear at exponent 0.9, inference at 0.6
tobyordoxford · x · 2026-11-14
Oxford researcher Toby Ord estimates that returns to pre-training compute are close to linear — his rough estimate is an exponent of 0.9 (odds ∝ FLOPs^0.9), sublinear but far better than logarithmic.
For inference compute, going from 20% to 80% success typically takes only 100x more compute, doubling the effective exponent to 0.6 (odds ∝ inferencecompute^0.6). The thread with Ramez Naam, Ben Todd and Allan Dafoe offers concrete numbers for the debate over how much capability more compute can still buy.
Related event: Oxford's Toby Ord: RL Compute Needs 10,000x Boost to Lift Benchmark Scores(17 posts)→
More from AGI Musings
- Interconnects: Jensen's resignation is a major setback for US open-model ecosystem — Interconnects (Nathan Lambert) · 2026-11-14
- AGI endgame: either a balanced multipolar world or a centralized panopticon — CedricMakes · 2026-11-14
- "60% of people around you have some form of AI psychosis," says Alsop III — StewartalsopIII · 2026-11-14
- AI won't make everyone equal—it will split the smart from the rest worse than money — prasenx · 2026-11-14
- Toddlers learn new verbs only from responsive partners; a call to rerun the test with AI — AnnaCiaunica · 2026-11-14
- Developer muses: in Murphy's world, billions of interdependent events make rare failures daily — bingxu_ · 2026-11-14