Cost Pareto-dominant, but second only to Astra on output-token efficiency
adonis_singh · x · 2026-09-30
- The author updated their leaderboard with new colors for readability.
- On cost, the model is Pareto-dominant.
- On output tokens, it ranks second only to Astra: at every reasoning effort level it sits entirely below Astra, meaning it's less efficient on this benchmark.
More from Models
- Google hides Gemini model label in AI Mode, rolls Gemini 4 Carbon to Pro subscribers — gaganghotra_ · 2026-11-16
- Open-weight Karpo-Decision-31B tops Decision Index at 73.9, one forward pass per answer — CodeByPoonam · 2026-11-16
- Stealth Anthropic model spotted in Devin build as METR chart shows frontier pace up 1.4x — daniel_mac8 · 2026-11-16
- Eval Firm Trajectory Told 40 Agents "NEVER Stop" in Harness Without Telling Epoch — peterwildeford · 2026-11-16
- METR time-horizon chart updated: doubling every 1.7 months since Opus 4.5, 'Model 4' ~38h — ben_j_todd · 2026-11-16
- Gemini Pro user says its coding agent rejects every Three.js/SVG prompt as 'illegal' — Necessary-Buy-7410 · 2026-11-16