Anthropic benchmark score looks fine, but high ECI makes small gaps look bigger
scaling01 · x · 2026-07-25
The thread says the score itself looks fine: it is reportedly better than Opus 4.8, but still below Fable. The confusing part, they argue, is that Anthropic’s ECI is so high.
They also note that because ECI is sensitive to one-point differences, even small gaps can look more dramatic than they really are.
Related event: Anthropic's Benchmark Scores Spark Community Trust Crisis(2 posts)→
More from Models
- User claims Kimi K3.1 Pro High generated an LLM explainer video; model name unverified — iamfakhrealam · 2026-10-27
- Anyone can get an abliterated model with safety training stripped out for pennies — EXM7777 · 2026-10-27
- Ask an LLM for 10 bold 2030 AI predictions — many already happened — FlorianGallwitz · 2026-10-27
- Prefilling probe shows curl trigger: 98% completions write a URL once '-o /tmp/' is appended — rickasaurus · 2026-10-27
- Unsloth details running DeepSeek-V4.1-Flash locally: 1-bit GGUF fits 128GB RAM with 1M context — gnukeith · 2026-10-27
- X Chat now shows your Grok weekly usage limit and reset time in the inbox — XFreeze · 2026-10-27