METR: AI agents progress as fast on costlier tasks as cheap ones, doubling times similar
CFGeek · x · 2026-11-21
METR released new agent evaluation data, with researcher slimshetty noting a counterintuitive finding: he expected progress to slow on fuzzier, costlier tasks, but that's not what the data shows.
Key points:
- For a given task length, agents do somewhat worse on costlier tasks
- However, progress rates (doubling times) are roughly the same across cheap and expensive task regimes, even for more real-world tasks
- Frontier performance on the costlier tasks is close to what OpenAI reports for its internal research use
The data does not support the hypothesis that capability gains stall on harder, pricier tasks.
More from Models
- GPT-6.2 Astra Tops RoboColosseum SO-ARM101 Leaderboard With 1,984 Elo — DJiafei · 2026-11-21
- Kaomoji as an emotion probe: Qwen3-8B's internal "happiness" is measurable — Sauers_ · 2026-11-21
- 27,262 thinking tokens, zero answer: Google auto-billed 23 empty Vertex AI crashes via HTTP 200 — Plenty-Jackfruit-218 · 2026-11-21
- Thinking Machines' new model claimed 5x faster than Qwen3.8, 15% more accurate — LiTianleli · 2026-11-21
- Gemini silently saves every Google search that gets an AI response, user finds — The-Neofighter · 2026-11-21
- Frontier Design Benchmark: Claude Fable 5.2 tops ranking, just 26 Elo below human designs — yuwen_lu_ · 2026-11-21