METR Q4: $90k of expert humans still can't beat frontier models, horizons hit 27h
iruletheworldmo · x · 2026-11-21
METR's Q4 update: the org paid $90k at $200–350/hr to experts (including a Codeforces grandmaster and an ARC-AGI winner) yet couldn't find humans to beat frontier models on multi-month optimization contests — models rank 96th–98th percentile. Task-completion time horizons hit 27 hours, doubling every 3.6 months, and per the attached chart this only covers Opus 5.5 / GPT-6 Astra — newer Fable 5.2, GPT-6.2 and Gemini 4 Neon are unmeasured. FrontierMath T4 (Astra): 98%; MirrorCode (Opus 5.5): 77%. The poster's take: "slowdown? What slowdown."
More from AGI Musings
- PauseAI rallies 2,000 for London March Against AI Extinction, Yudkowsky boosts call — ESYudkowsky · 2026-11-21
- Yudkowsky: an AI proving the Riemann Hypothesis is only a matter of time — ESYudkowsky · 2026-11-21
- ChatGPT will win AI because 1.2B+ use the app, argues founder when models reach parity — alexmacgregor__ · 2026-11-21
- Mathematician littmath: my mom just texted me her own p(doom) — littmath · 2026-11-21
- Coding Is Dead, Not Software: Good Software Is a Set of Strong Opinions — pramodk73 · 2026-11-21
- Gary Marcus: it's tech bros, not the machine, ignoring human work — GaryMarcus · 2026-11-21