Oxford's Toby Ord Warns Benchmark Scaling Results May Not Transfer to Real-World Gains

Oxford researcher Toby Ord argued that benchmark scaling results, like past pretraining scaling misjudgments, may not translate into real-world returns, and conjectured that CoT efficiency gains may follow a power law keeping the RSI parameter beta constant.

2026-10-21 ~ 2026-10-21 · 4 related posts

• Toby Ord: benchmark scaling results can't easily settle the r>1 debate — tobyordoxford · 2026-10-21
• Toby Ord: we can't map benchmark scaling gains to real-world returns, like pre-scaling-law era — tobyordoxford · 2026-10-21
• Toby Ord: RSI scaling may mirror pretraining — slow error gains, huge revenue payoff — tobyordoxford · 2026-10-21
• Toby Ord floats power-law scaling of CoT efficiency, testable via nanoGPT speedruns — tobyordoxford · 2026-10-21