METR: Anthropic's AI progress accelerating at 1.9x, driven by AI-assisted research
mark_k · x · 2026-11-07
After an 11-week high-access evaluation at Anthropic, METR estimates the company's models have been improving on well-specified benchmarks at roughly 1.9x the pre-October-2025 rate (95% CI: 1.5–2.8x) — about two years of historical progress in one year.
- Evaluators accessed internal model results, training metrics (pretraining, RL, per-domain compute) and interviewed staff
- METR attributes most of the acceleration to AI assisting human researchers, with fully autonomous contributions still limited; faster training-compute scaling was ruled out
- The report flags a lack of observed bottlenecks or diminishing returns, suggesting risk of even faster acceleration, while acknowledging possible missed bottlenecks
- Estimates carry high uncertainty (true rate could be closer to 1.3x or 3x); Anthropic redacted supporting evidence from the public report over competitive concerns
More from Models
- From "can't write coherent text" to complaining a 2B model can't run a training pipeline in 10 minutes — cephaloform · 2026-11-07
- Rubricon: scaling test-time compute to auto-generate expert-grade LLM rubrics — deliprao · 2026-11-07
- Kimi K3.1 Pro lands on Microsoft Foundry with 1M-token context for agentic work — DanWahlin · 2026-11-07
- Ripple 65B Preview vs Qwen 3.8 coding benchmarks: 79.1 on SWE-Bench Pro — Nota_ReAlperson · 2026-11-07
- Fireship: OpenAI's unnamed model just cracked 3-SAT, threatening a 27-year-old hypothesis — Fireship · 2026-11-07
- Bittensor subnet model tops respected independent leaderboard just ~30 days after launch — airesearch12 · 2026-11-07