METR sensitivity analysis: AI capability slope increase often vanishes by method
tobyordoxford · x · 2026-11-08
- Oxford researcher Toby Ord discusses METR's AECI evidence: the team ran sensitivity analyses over different ways of computing ECI and which data to plot.
- The chart matrix shows slope estimates for 2025→2026 ranging from 0.99× to 1.90× depending on methodology; under several specifications the slope-increase effect in central estimates disappears entirely.
- Ord argues evidence for any slope increase is weak: central points do rise, but they sit at the upper range of available benchmarks with correlated error — and there simply isn't much public evidence, which isn't METR's fault.
More from AGI Musings
- The brain as the next UI: upcoming article charts BCI-meets-agentic-AI roadmap via earbuds — melnykowycz · 2026-11-08
- AI helps neurosurgery resident settle 22-year-old Crouzeix's conjecture in matrix analysis — IgorCarron · 2026-11-08
- NY Assemblyman urges Congress to ban Recursive Self-Improvement this year, citing 2027-28 AI lab timelines — rohanpaul_ai · 2026-11-08
- Radziwill says the quasi-Riemann proof fits in a tweet from the future — Singularitarian · 2026-11-08
- DeepMind exec says scientific discovery is undergoing a 'genuine phase transition' in pace — pushmeet · 2026-11-08
- IDC analyst: AI claims on social media distort completely after two reshares — perilli · 2026-11-08