Anthropic benchmark score looks fine, but high ECI makes small gaps look bigger
scaling01 · x · 2026-07-25
The thread says the score itself looks fine: it is reportedly better than Opus 4.8, but still below Fable. The confusing part, they argue, is that Anthropic’s ECI is so high.
They also note that because ECI is sensitive to one-point differences, even small gaps can look more dramatic than they really are.
Related event: Anthropic's Benchmark Scores Spark Community Trust Crisis(2 posts)→
More from Models
- Kimi K2.8 Preview rolls out: near-K3 coding performance, 1M context for all tiers — teortaxesTex · 2026-09-11
- Looking for a classifier of software engineering task shapes to pick models per task — StewartalsopIII · 2026-09-11
- DeepSeek V4 Pro API to continue after Sept 2026, billing unchanged — teortaxesTex · 2026-09-11
- DeepSeek V4.1 Flash Hits 98% of GPT-6 Astra's Score at 1.4% of the Cost in Third-Party Benchmark — ayushtweetshere · 2026-09-11
- TheZvi Polls: Has Your Coding Model Choice Changed Since Fable 5.1 and Astra? — TheZvi · 2026-09-11
- antirez Weighs In on Anthropic Banning Minors From Using Claude — antirez · 2026-09-11