Oxford's Toby Ord: OpenAI's real-world R&D time horizon is 1/12 of METR benchmark

On October 22, Oxford scholar Toby Ord published a series of analyses on the gap between METR's time horizon benchmark and OpenAI models' real-world performance. His core conclusion: under METR's 80%-success-rate task-length standard, from January to July 2025 OpenAI models' average time horizon on real-world AI R&D tasks was only about 1/12 of the METR task-suite level—equivalent to falling roughly 3.6 doublings, or about 14 months, behind the prevailing fast exponential trend.

Confirmed

• Ord ran a sanity check on METR's time horizon charts: if real-world AI R&D work is assumed to be as hard as the METR task set, then OpenAI's published numbers imply actual progress is overstated by about 14 months relative to what the charts show.
• Ord cited @rameze's analysis of the current state of recursive self-improvement, quoting a key figure from OpenAI's own report on research automation: the model's 80% time horizon on R&D tasks is only 15 minutes, meaning models struggle to sustain long, independent work on a task.
• Ord contrasted the data: METR's "80%-success time horizon" metric (the length of human-time tasks an AI can complete with 80% probability) had reached 3 hours by the time reporting stopped in April 2026, while OpenAI's internal figure was just 15 minutes.

Why it matters

• The METR time-horizon benchmark is widely used as a core metric for frontier models' ability to autonomously complete long tasks, and is extrapolated into fast exponential progress trends; if the real R&D time horizon is only 1/12 of the benchmark, capability extrapolations and automation timelines based on it may be significantly optimistic.
• OpenAI's own reported 15-minute 80% time horizon offers an internal view of the current state of recursive self-improvement, in stark contrast to benchmark scores.
• Ord's analysis is an inference from public data, resting on the implicit assumption that real-world R&D tasks are comparable in difficulty to the METR task set—this assumption is key to whether the conclusion holds.

2026-10-22 ~ 2026-10-22 · 8 related posts

• OpenAI's own report shows models' 80% time horizon on R&D is just 15 minutes — tobyordoxford · 2026-10-22
• METR hit 3-hour time horizons by April 2026, but OpenAI's own R&D tasks show only 15 minutes — tobyordoxford · 2026-10-22
• OpenAI models' task horizon just 1/12 of METR trend — about 14 months behind, says Toby Ord — tobyordoxford · 2026-10-22
• [source] OpenAI models' real-world AI R&D time horizon is only 1/12 of METR's benchmark suite — tobyordoxford · 2026-10-22
• Toby Ord sanity-checks METR time horizons: OpenAI's number may run 14 months behind expectations — tobyordoxford · 2026-10-22
• Toby Ord: Gemini 4 Safety Report discloses DeepMind's internal AI-for-AI-R&D efficiency data — tobyordoxford · 2026-10-22
• [source] Gemini 4 safety report shows AI hits 80% reliability on only ~15-minute AI R&D tasks — tobyordoxford · 2026-10-22
• [source] Ben Todd: real research tasks are only ~1 year behind the METR time-horizon — ben_j_todd · 2026-10-22