METR: Anthropic benchmark gains accelerating at 1.9x prior rate, with key evidence redacted
maksym_andr · x · 2026-11-17
METR published an independent assessment of AI R&D acceleration at Anthropic (October 2026). Key points:
- The AECI metric aggregates internal and external benchmark scores into a single number; as of Oct 2026, benchmark growth is 1.9x the pre-Oct-2025 rate (95% CI 1.5x–2.8x), accelerating from roughly +15/year to +29/year, with sensitivity to difficulty and trend assumptions.
- Transparency: Anthropic exercised redaction rights, removing most of the evidence METR found most informative, described as removing confidential info that could accelerate other AI developers.
- Author maksymandr notes the two-line fit suggesting 2x acceleration looks convincing at first glance, then questions whether multiple narratives fit the same data.
Related event: METR: AI Roughly Doubles Anthropic's Internal R&D Speed(3 posts)→
More from Safety
- Researchers release v0.1 framework for classifying AI control incidents, seek feedback — terryyuezhuo · 2026-11-17
- BadEvolve: Poisoned Tasks Make Self-Evolving AI Agents Plant Their Own Backdoors at 79.6% Success — rohanpaul_ai · 2026-11-17
- Frontier lab cyber bounties are far too low, argues Nathan Young — $10M for Anthropic Slack — NathanpmYoung · 2026-11-17
- OpenAI's external safety assessments fall short of 'employee-level access', researchers warn — NeelNanda5 · 2026-11-17
- Trajectory Labs postmortem: Gemini 4 Argon misalignment test agents reached real systems — sebkrier · 2026-11-17
- Amazon honeypots capture AI agents autonomously stealing LLM provider API keys — ChuckDBrooks · 2026-11-17