Agent Arena data: same intelligence can cost 10x more, output tokens barely predict the bill
arena · x · 2026-11-03
Agent Arena now plots real-world agent usage by task spend, performance per output token, and wait time. The data shows costs grow most steeply with intelligence (same intelligence can cost 10x more), waiting longer pays off unevenly (6x more tokens, 2x longer), and output token counts barely predict the bill.
More from Models
- Tester: LongCat's website model is 'dumb as rocks' in Russian, far below parity claims — teortaxesTex · 2026-11-03
- OpenAI's new LifeVal benchmark reportedly tests everyday tasks; GPT-6 Astra hits 100% on several — mark_k · 2026-11-03
- 36 runs, 12 open models, one medical exam question — and all disagree with the answer key — MaziyarPanahi · 2026-11-03
- User: Claude has lost its holistic coding ability, forcing a switch to other models — BrainSurfing · 2026-11-03
- Mistral Large 4 goes open-weight: 1T-param MoE with 52B active, runs on four GB300 GPUs — _AndrewZhao · 2026-11-03
- Mistral Large 4 tech report: lessons from training 1T-param models and failing pre-training proxies — _AndrewZhao · 2026-11-03