Agent Arena Ranks 43 Models on 2M+ Real-World Agentic Tasks; Claude Fable 5.1 Tops Board
arena · x · 2026-09-25
Agent Arena launched a leaderboard measuring models on millions of real-world, long-horizon agentic tasks from a global user community, using causal tracing to compute net improvement vs. the average model. Models get web search, filesystem, and terminal tools.
Key rankings:
- Claude Fable 5.1 (Max) leads at 13.44% net improvement, $4.15 median cost/task
- GPT 6 Astra (Max) follows at 11.08%, $2.86/task
- Claude Opus 5 (High/Max) ranks 3rd and 4th
- Open-source Kimi K3 (Max) enters the top 9 at 4.56%
Metrics include confirmed success rate, praise vs. complaint, steerability, bash recovery, and tool hallucination rate.
More from coding & agent
- Spotify's Claude Code 90% token-saving trick put to a real test — aakashgupta · 2026-11-11
- Bolt: Gemma 4 runs 100% on-device on iPhone to drive a robot dog health companion — DynamicWebPaige · 2026-11-11
- Dev shows off live voice-call AI agent 'Elise' that answers family calls and triages urgency — bigaiguy · 2026-11-11
- agents.md isn't just for AI agents — humans can grok codebases fast with it too — k7agar · 2026-11-11
- Animate real places in 3D with Claude and free map data from AWS and Overture — drbronk · 2026-11-11
- Microsoft Foundry pattern: version and deploy Toolboxes separately from agents behind stable endpoints — WirelessLife · 2026-11-11