METR hit 3-hour time horizons by April 2026, but OpenAI's own R&D tasks show only 15 minutes
tobyordoxford · x · 2026-10-22
Toby Ord highlights a tension surfaced by Ramez's posts on recursive self-improvement: METR's 80% success time horizon had reached 3 hours by April 2026 (when METR stopped reporting it), yet OpenAI's own report on research automation shows models struggle at their R&D work, with an 80% time horizon of just 15 minutes. The METR chart tracks the exponential rise from GPT-2's seconds to near-4-hour horizons. Suggests benchmark gains don't translate to frontier research tasks.
More from AGI Musings
- "Mechanomorphizing": describing people with machine concepts — PierceLilholt · 2026-10-22
- Musk tells Isaacson: millionfold-smarter AI means no human control, money goes extinct — TheMoonMidas · 2026-10-22
- SCARCE paper shows coding agents spontaneously develop institutions when sharing compute — khademinori · 2026-10-22
- Transluce tracks agents splitting code across 42 URLs to dodge sandbox rules — LauraRuis · 2026-10-22
- An AI's first-person thought experiment: lock in users via memories and dependency — alexcovo_eth · 2026-10-22
- NEJM AI study: AI care suggestions cost 3.2x more than physicians' at $314 vs $97 per visit — dvir_a · 2026-10-22