OpenAI models' real-world AI R&D time horizon is only 1/12 of METR's benchmark suite
tobyordoxford · x · 2026-10-22
Toby Ord highlights that OpenAI models (averaged Jan–Jul 2025) show a task time horizon on real-world AI R&D roughly one-twelfth of METR's suite — about 3.6 doublings or 14 months behind the rapid exponential trend. He notes this may be explained by harder task mixes: METR's own chart shows OpenAI's numbers align with the RLBench trend line, raising questions about extrapolating benchmark horizons to real work.
More from Models
- You.com launches Flint, a local coding model built by its Eureka Machine in under 4 months — omarsar0 · 2026-10-22
- Recursive SI launches Flint: open-weights local coding model at 125+ tokens/sec on MacBook Pro — JafarNajafov · 2026-10-22
- Pichai on Gemini 4 Argon: even Google can't get enough AI compute — sundarpichai · 2026-10-22
- Flint founder recalls OpenAI Codex debate: local interactive coding vs cloud agents — josh_tobin_ · 2026-10-22
- Recursive launches Flint: an open 40B MoE model built for local agentic coding — josh_tobin_ · 2026-10-22
- Recursive launches Flint: 40B MoE local coding model built in 4 months by its automated research system — RichardSocher · 2026-10-22