METR: Anthropic models improving 1.9x faster since Oct 2025, though estimate hinges on choices
NathanpmYoung · x · 2026-11-06
METR's new evaluation finds Anthropic's AECI scores have grown 1.9x faster (95% CI: 1.5–2.8) since October 2025. Key takeaways from the commentary: evidence comes mainly from an expanded version of Anthropic's internal ECI plus researcher reports, with other evidence redacted (likely algorithmic efficiency gains); Anthropic no longer releasing its best models has made public ECI less useful; adding historical models from other providers drops the speedup to 1.4x since Anthropic lagged the frontier earlier in 2025; researchers report large uplift from research agents. METR cautions that benchmarks are only one piece of evidence and their mapping to real-world capability remains unclear.
More from AGI Musings
- Commentary: AI CEOs have sucked at messaging why AI is good — AIandDesign · 2026-11-06
- AIFP warns: if Anthropic hits its 2x RSP acceleration threshold, superintelligence timelines may collapse — austinc3301 · 2026-11-06
- Grok and ChatGPT agree intelligence has limits; author predicts peak by 2030 — davidpattersonx · 2026-11-06
- World models can 'see into' any drawing, photo or render: a computer graphics renaissance — XRarchitect · 2026-11-06
- Eli Lifland: A 2x Progress Rate by Dec 2026 Means Roughly 4.5x Capability Uplift — inductionheads · 2026-11-06
- Rethinking predoc interviews in the AI era: agent transcripts and blindfolded video tests — aniketapanjwani · 2026-11-06