Oxford's Toby Ord: RL Compute Needs 10,000x Boost to Lift Benchmark Scores

Oxford scholar Toby Ord, in a discussion with Ramez, Ben Todd and others on the relationship between RL compute scaling (RLcompute) and task horizons, offered a set of theoretical arguments whose core conclusion is: the length of tasks a model can complete may grow exponentially with effective RL compute.

Confirmed

• The ECI (Extended Capability Index) applies an inverse logistic transform to each benchmark first, straightening the S-shaped (logistic) growth in probabilities into linear growth. This is the mathematical construction of the metric, which Ord explicitly explained.
• Ord noted that benchmark scores (percentages) are essentially success probabilities on random tasks, and can be rewritten as odds for analysis; when success rates grow logistically over time, expressing them in odds means the odds are rising exponentially.
• From this he derived: linear growth in ECI corresponds to exponentially growing capability, and task horizons behave like an expansion process with a constant hazard rate—i.e., horizons stretch exponentially with effective compute (or time).
• He also responded to criticisms of ECI: critics argue that the log/inverse logistic transform makes logistic growth "look linear" and may exaggerate the trend; Ord countered that this transform is the mathematically natural way to analyze a logistic curve, and from the odds perspective the growth is indeed exponential.

Not Yet Confirmed

• The exponential expansion of horizons with RL compute is a theoretical argument derived from the metric's mathematical properties. Whether it holds on actual benchmark data still awaits empirical validation, and the discussion included skepticism that the ECI transform may distort trend judgments.

Why It Matters

• If task horizons truly grow exponentially with RL compute, it means the length of tasks AI can autonomously complete will advance much faster than linear projections, with direct implications for AI capability forecasting and safety planning. The debate also highlights methodological controversies around aggregate metrics like ECI—the choice of transform itself can shape how people judge the pace of capability growth.

2026-11-14 ~ 2026-11-14 · 17 related posts

• Toby Ord: task horizon likely follows a power law in RL compute, not linear — tobyordoxford · 2026-11-14
• [source] Toby Ord argues benchmark odds give theoretical basis for exponentially growing task horizons — tobyordoxford · 2026-11-14
• Toby Ord argues odds reformulation explains why AI task horizons grow exponentially with ECI — tobyordoxford · 2026-11-14
• [source] Toby Ord: Logistic Benchmark Growth Means Success Odds Rise Exponentially Over Time — tobyordoxford · 2026-11-14
• Toby Ord defends ECI benchmark: inverse-logistic transform exposes exponential odds growth — tobyordoxford · 2026-11-14
• [source] Toby Ord: Linear ECI Growth Means Exponential Growth in Task-Horizon Odds — tobyordoxford · 2026-11-14
• Toby Ord: expected consecutive successes before failure exactly equals the odds of success — tobyordoxford · 2026-11-14
• Toby Ord: Elo grows linearly but winning odds grow exponentially — odds are the real AI progress metric — tobyordoxford · 2026-11-14
• Toby Ord: Elo and odds are the same measure on a log scale — ECI works the same way — tobyordoxford · 2026-11-14
• Toby Ord explains why ECI and benchmark success odds are a log-transform pair — tobyordoxford · 2026-11-14
• Toby Ord plots capability vs log(compute), assumes logistic scaling curves — tobyordoxford · 2026-11-14
• Toby Ord: task success odds grow as a power law in compute, logistic curves need checking — tobyordoxford · 2026-11-14
• RL vs pretraining compute have different power-law exponents on benchmarks — tobyordoxford · 2026-11-14
• Toby Ord: RL, inference, and pretraining compute show different scaling exponents in preliminary analysis — tobyordoxford · 2026-11-14
• Toby Ord: RL needs ~10,000x compute to go from 20% to 80% on reasoning benchmarks — tobyordoxford · 2026-11-14
• Oxford's Toby Ord estimates pre-training compute returns near-linear at exponent 0.9, inference at 0.6 — tobyordoxford · 2026-11-14

1 near-duplicate retellings: tobyordoxford