Toby Ord: benchmark scaling results can't easily settle the r>1 debate
tobyordoxford · x · 2026-10-21
Toby Ord (Oxford) argues in a thread with Ramez, Tom Davidson and others that building a case for returns r>1 from benchmark scaling results has a huge gap: we have little ability to pin down the function mapping efficiency gains to benchmark scores, so results don't transfer cleanly to real-world scaling.
He suggests treating efficiency gains as a function of benchmark performance that could be super-linear, perhaps even exponential — training and inference efficiency at fixed performance has shown super-linear returns to researcher time — but that doesn't imply real-world performance only scales logarithmically with these inputs.
More from AGI Musings
- When AI saves you an hour at work, who decides where that hour goes? — Alchemistwiza · 2026-10-21
- Quantum+AI researcher asks: are we optimizing for the right goals? — AryHHAry · 2026-10-21
- Nadella: SaaS Will Collapse in the Agent Era — But One Rebuild Cost More Than 2 Years of Subscription — msharmas · 2026-10-21
- AI May Soon Make Scientific Breakthroughs Nobody Can Explain, warns AI commentator — VraserX · 2026-10-21
- Mathematicians clash: do self-maintaining biological neurons make AI fundamentally different? — ValerioCapraro · 2026-10-21
- "Every job was a placeholder until a machine could do it" — bold AGI labor take — rand_longevity · 2026-10-21