Anthropic models burn 2-3x tokens and cost 10-50x more on FrontierCode 2.0
teortaxesTex · x · 2026-10-28
An overlooked observation from the FrontierCode 2.0 benchmark (Cognition's 132-task agentic coding eval graded on real open-source repos): Anthropic models don't care about token efficiency.
Per Artificial Analysis V5 data:
- For frontier-complexity coding tasks, Anthropic models are the clear choice — but Claude models generate 200-300% more tokens (test-time compute) on Frontier Code 2 tasks
- They take twice as long and cost 10-50x more
- Open models retain favorable economics and may reach frontier performance with more test-time compute
The attached Artificial Analysis chart shows token usage distribution across the Coding Agent Index: Claude Sonnet averages 18.9M tokens per task, vs 9M (DeepSeek), 8.1M (GLM-5.3), and as low as 2.2M (GPT-6 Astra). The poster also praises GPT-6.1-Terra's efficiency and suspects a distillation breakthrough.
Related event: FrontierCode 2.0 leaderboard lands; hidden model Obsidian 3 tops the chart(10 posts)→
More from coding & agent
- Drew Breunig's State of Context Engineering: 6 tactics, one key tradeoff — rseroter · 2026-10-29
- GitHub demos Copilot Code Review Max with stacked PR merge at Universe — kimmonismus · 2026-10-29
- IBM TechXchange: Three Big Challenges of Adopting AI Agents in the Enterprise — craigmullins · 2026-10-29
- Pydantic Author Samuel Colvin on Setting Up Feedback Loops for Code-Writing Models — samuelcolvin · 2026-10-29
- Open-Source Coding Tools Climb T3 Charts: opencode Hits #2 Most Engaged Repo — MeetPatelTech · 2026-10-29
- Dev builds a GitHub knowledge layer: MCP server indexing the 60% of knowledge hidden in comments — jamesmurdza · 2026-10-29