Anthropic benchmark score looks fine, but high ECI makes small gaps look bigger
scaling01 · x · 2026-07-25
The thread says the score itself looks fine: it is reportedly better than Opus 4.8, but still below Fable. The confusing part, they argue, is that Anthropic’s ECI is so high.
They also note that because ECI is sensitive to one-point differences, even small gaps can look more dramatic than they really are.
Related event: Anthropic's Benchmark Scores Spark Community Trust Crisis(2 posts)→
More from Models
- GPT-6.1 Terra shows off Blender 3D computer use, using just 3% of a Pro 200 plan — sandersted · 2026-10-27
- Google DeepMind exec casually confirms 'Gemini 4' exists in X reply — DeryaTR_ · 2026-10-27
- Gemma hits 2.85x speedup on vLLM: 37,900 to 108,000+ tokens/sec on one A100 — patloeber · 2026-10-27
- Claude Duet has no credential store: can't securely hold API keys or call APIs — altryne · 2026-10-27
- Radiology's Last Exam 2.1 teaser: leaderboards flip as Gemini 4 Argon tops health index — DrDatta_AIIMS · 2026-10-27
- Reddit Users Say the Ming Image Model Is Seriously Underrated — alley103 · 2026-10-27