METR: Anthropic likely improving model capabilities ~2x faster than historical rate
scottleibrand · x · 2026-11-06
METR, working with internal Anthropic access since August, published its assessment of whether AI is accelerating AI development. Despite significant uncertainty and internal disagreements, most METR staff on the project believe that as of October 2026, Anthropic is likely improving model capabilities on well-specified tasks at roughly twice its historical (March 2024–October 2025) rate — a 1.9x AECI acceleration (95% CI: 1.5–2.8x); they wouldn't be surprised if the true rate were 1.3x or 3x.
- Beyond AECI data, redacted internal evidence made the team take the ≈2x acceleration seriously, with indications it could exceed 2x and increase over time — though significantly higher would surprise them.
- Caveats: the AECI acceleration could reflect benchmark overfitting or methodological artifacts, and AECI data lags and is sparse, limiting its ability to estimate recent R&D acceleration.
- Uncertainty remains on how this acceleration translates to broader, less well-specified tasks.
More from AGI Musings
- METR: No clear bottlenecks seen, capability progress could accelerate further in 12 months — paulnovosad · 2026-11-06
- Greg Brockman: AI to drive major biology, health breakthroughs in 12-18 months — ChrisGPT · 2026-11-06
- Ethan Mollick: an AI stock downturn won't derail the AI trajectory — emollick · 2026-11-06
- Model value as conditional entropy removed: why rising world entropy hurts model companies — xiaosun86 · 2026-11-06
- The most valuable secrets in AI are no-go theorems, not new tricks — fkasummer · 2026-11-06
- GitHub commits hit 275M/week as AI-generated tokens commoditize frontier labs' edge — DimitrisPapail · 2026-11-06