DeepMind alignment researcher signs open letter urging coordinated AI slowdown
vkrakovna · x · 2026-09-11
Alignment researcher Victoria Krakovna announced she signed a letter calling for building capacity for a coordinated slowdown of frontier AI development.
Her core argument:
- A runaway race to AGI is unsafe: it incentivizes cutting corners on safety, and model capabilities risk outpacing alignment, control, and governance measures, potentially causing catastrophic loss of control
- Slow, controlled capability increases would allow developing and testing alignment methods for each capability level; advanced AI should only be built with robust safety assurance
She has previously stated, in a personal capacity, a >10% chance of advanced AI causing human extinction within a decade, and currently works on honeypots to catch scheming AI.
More from AGI Musings
- Ex-OpenAI/Anthropic pretraining researcher resigns, citing reckless ASI race — oh_that_hat · 2026-09-11
- Eric Topol and JAMA AI editor discuss what it takes for AI to shift medicine to prediction and prevention — EricTopol · 2026-09-11
- The barbell strategy for thriving in a post-AGI world: double down on AI and on being human — brandon_galang · 2026-09-11
- "A statistical database can't end humanity" — viral rebuttal of AI doom, retweeted by Gary Marcus — GaryMarcus · 2026-09-11
- Perry Metzger: Build Formal Verification for AI Security Instead of Panicking — jd_pressman · 2026-09-11
- Statistician Kareem Carr: AI safety arguments must show what's uniquely dangerous about AI — kareem_carr · 2026-09-11