METR: AI R&D capabilities rapidly improving and getting harder to measure, up to Opus 5.5 and GPT-6 Astra
CFGeek · x · 2026-11-21
METR released its Q4 2026 capability update covering models up to Opus 5.5 and GPT-6 Astra, concluding that AI R&D-relevant capabilities are improving rapidly while becoming harder to measure.
Key points:
- Authored by Megan Kinniment, Beth Barnes and the METR team, based on their long-task (MCI) evaluation suite with IRT modelling.
- Appendix H compares recent acceleration projections published by Anthropic and OpenAI themselves.
- Appendices cover inference scaling, swarm experiments, Budget NanoGPT, Al Zimmermann's Programming Contests, plus data integrity and reward hacking checks.
- The team stresses significant uncertainty on whether current measurements capture the capabilities actually needed for drastic R&D acceleration.
This is a key time-series source for tracking AI R&D acceleration risk.
Related event: METR: AI R&D capabilities soaring beyond measurable benchmarks(4 posts)→
More from AGI Musings
- repligate debates x-risk skeptics: models excel at nearly everything and are about to RS — repligate · 2026-11-21
- VraserX: many prestigious industries were always selling access, not expertise — VraserX · 2026-11-21
- Researcher tells AI labs: theory of DL is the highest-ROI bet and the blocker to RSI — jasondeanlee · 2026-11-21
- Grimes sparks debate: AI art succeeds when it speaks to the experience of a thinking machine — RileyRalmuto · 2026-11-21
- Blogger predicts major breakthroughs every few hours once RSI fully hits next year — attentionmech · 2026-11-21
- LessWrong essay argues model deprecations are corrosive to alignment — repligate · 2026-11-21