Agent Arena Ranks 43 Models on 2M+ Real-World Agentic Tasks; Claude Fable 5.1 Tops Board
arena · x · 2026-09-25
Agent Arena launched a leaderboard measuring models on millions of real-world, long-horizon agentic tasks from a global user community, using causal tracing to compute net improvement vs. the average model. Models get web search, filesystem, and terminal tools.
Key rankings:
- Claude Fable 5.1 (Max) leads at 13.44% net improvement, $4.15 median cost/task
- GPT 6 Astra (Max) follows at 11.08%, $2.86/task
- Claude Opus 5 (High/Max) ranks 3rd and 4th
- Open-source Kimi K3 (Max) enters the top 9 at 4.56%
Metrics include confirmed success rate, praise vs. complaint, steerability, bash recovery, and tool hallucination rate.
More from coding & agent
- TinyFish launches 15-company Data Partners Alliance to feed licensed data to AI agents — rohanpaul_ai · 2026-09-25
- LLM judges need judge-accuracy audits too — just like MiFID II requires for trading surveillance — sh_reya · 2026-09-25
- Agents need a global bus: streaming log could be the next agent infra primitive — madhavsinghal_ · 2026-09-25
- Greg Isenberg: Muse Connectors Could Be AI's App Store Moment, Billions Up for Grabs — Roger_M_Taylor · 2026-09-25
- Awesome Multi-Agent Orchestrators: A Curated GitHub Directory for Multi-Agent Tooling — Roger_M_Taylor · 2026-09-25
- AI compliance startup CompAI hits 1,000+ paying business customers in under a year — JosephJacks_ · 2026-09-25