METR: AI nearly doubles research progress; Ajeya Cotra warns industry unprepared
AndyMasley · x · 2026-11-07
Former OpenAI researcher Ajeya Cotra cites the METR AI R&D report's estimate that AI is nearly doubling the rate of capabilities progress — equivalent to researchers experiencing time 10x faster, unattainable even by hiring every qualified human.
- She argues this acceleration is a strong indicator of extreme capabilities very soon: Anthropic's May 2025 RSP set its top AI R&D-5 threshold at a weaker milestone, which would already require security strong enough to protect model weights from state-level adversaries
- She says the industry is nowhere near that bar — OpenAI and Anthropic both recently had major security breaches by amateur groups — and lists what an intelligence explosion would demand: robust weight security, universal logging/monitoring, monitorability measurements, anti-collusion countermeasures, convincing alignment training and interpretability, and governance for pacing progress
Related event: METR Report Finds Anthropic's Research Progress Doubling Yearly(2 posts)→
More from AGI Musings
- Stanford's Anshul Kundaje: ML conference peer review is broken and unfixable — anshulkundaje · 2026-11-07
- Eric Drexler: AI options, not optimism — participants weigh options, not odds — sebkrier · 2026-11-07
- Musk lays out SpaceX vision of deploying up to 1 TW of space-based AI compute per year — XFreeze · 2026-11-07
- "AI media detection will become an arms race just like cybersecurity" — ZeroStateReflex · 2026-11-07
- tszzl: Nature destroyed itself five times — stop reading morality into it — tszzl · 2026-11-07
- Vitalik Buterin: 'Open weight is not open source' on AI models — wonderwomancode · 2026-11-07