Agent Arena Launches Real-World Agent Leaderboard: Claude Opus 5 Tops, Kimi K3 Best Value
Agent Arena launched a leaderboard evaluating AI agents on real-world tasks, drawing on millions of long-horizon tasks submitted by a global community — 1.87 million sessions across 49 models. Agents could use tools such as web search, the file system and the terminal; the ranking applies causal tracing to measure net improvement over the average baseline across tool reliability, task completion and controllability, and pairs performance with per-task cost.
Confirmed
- Overall performance: Claude Opus 5 (High) ranks first with a 12.47% net improvement rate, followed by Kimi K3 (Max) and GPT 5.6 Sol (xHigh).
- Cost and value: Kimi K3 (Max) ranks fourth overall with a median task cost of $0.62, offering exceptional value; Claude Opus 5 (Max) scores slightly higher but costs $3.37 per task — the priciest in the top five — meaning agents in the same tier differ by more than 5x in task cost.
- Pareto-optimal: Claude Opus 5 (High) sits on the efficiency frontier at $1.78 per task with +12.34% net improvement.
Why it matters
- Compared with synthetic benchmarks, causal-tracing evaluation on real user tasks better reflects how models perform and stay controllable in complex workflows.
- By presenting performance and cost side by side, the leaderboard exposes huge gaps in "net improvement × per-task cost" among top models, giving practitioners a direct cost-effectiveness reference for model selection.
2026-08-20 ~ 2026-08-21 · 5 related posts
Primary sources
- Agent Arena: Kimi K3 offers best value, Claude Opus 5 is priciest — arena · 2026-08-20
- [source] Agent Arena: Top Agent Models Differ Over 5x in Cost per Task — arena · 2026-08-20
- [source] Agent Arena: Million-task leaderboard for real-world AI agents — arena · 2026-08-20
- [source] Agent Arena Leaderboard: Claude Opus 5 Takes Top Spot — arena · 2026-08-20
1 near-duplicate retellings: arena