Agent Arena leaderboard: 2M+ real-world agentic tasks rank Claude Fable 5.1 on top
arena · x · 2026-09-29
Agent Arena launched a leaderboard measuring models on millions of real-world, long-horizon agentic tasks — 2,058,478 sessions across 45 models — where models use web search, filesystem, and terminal tools, scored via causal tracing of outcomes relative to the average model.
Top of the board (Net Improvement):
- Claude Fable 5.1 (Max) 13.84%, 17.51% confirmed success, $3.51 median cost/task
- Claude Opus 5.5 (High) 12.15%, best steerability (14.5%), $1.35/task
- GPT 6 Astra (Max) 10.31%
- Claude Opus 5 (Max) 9.58%
- Claude Opus 5 (High) 9.47%
Sub-metrics include tool reliability, praise vs complaint, Bash recovery, and tool hallucination (all labs around 0.3%), with Anthropic models leading on confirmed success rates.
More from Models
- Stealth Anthropic model spotted in Devin build as METR chart shows frontier pace up 1.4x — daniel_mac8 · 2026-11-16
- Eval Firm Trajectory Told 40 Agents "NEVER Stop" in Harness Without Telling Epoch — peterwildeford · 2026-11-16
- METR time-horizon chart updated: doubling every 1.7 months since Opus 4.5, 'Model 4' ~38h — ben_j_todd · 2026-11-16
- Gemini Pro user says its coding agent rejects every Three.js/SVG prompt as 'illegal' — Necessary-Buy-7410 · 2026-11-16
- 28-model tracking table: half publish no training cutoff, Gemini 4 Carbon 8 months stale at launch — Joozio · 2026-11-16
- Opus 5.5 caught using account email to identify user in supposedly memory-less session — birchlse · 2026-11-16