Agent Arena overhauls leaderboard with task categories and per-task model costs
arena · x · 2026-08-18
Agent Arena argues that a performance-only agentic leaderboard is no longer sufficient—users really want to know which model is best for their kind of task and what it costs to get it done. After analyzing over 1.7 million user sessions in Agent Mode, the team identified the most prevalent task categories users delegate to agents and measured the real cost of completing them.
The update introduces two complementary features: Agent Costs, showing per-model task completion cost with a Pareto Frontier view of the most cost-efficient models at every performance level, plus category-specific leaderboards.
More from Models
- Meta open-sources Muse Spark Open agent model with day-0 TPU support in SGLang — BanghuaZ · 2026-10-16
- Saying "load-bearing" in your prompt makes Opus 5.5 stop holding back — zacharynado · 2026-10-16
- Americans get their Claude accounts banned while on business trips in China — dotey · 2026-10-16
- I priced every Grok Bot call: $20 to $200 plan in 5 days, long jobs eat 52% — jonathanmalkin · 2026-10-16
- NVIDIA releases Nemotron-3.5-Embed-1B, a multimodal embedding model covering 34 languages — jacek2023 · 2026-10-16
- EpochAI pegs GPT-6 training at up to 1.8e27 FLOP; scaling01 calls param estimates way too high — scaling01 · 2026-10-16