DeepMind alignment researcher signs open letter urging coordinated AI slowdown
vkrakovna · x · 2026-09-11
Alignment researcher Victoria Krakovna announced she signed a letter calling for building capacity for a coordinated slowdown of frontier AI development.
Her core argument:
- A runaway race to AGI is unsafe: it incentivizes cutting corners on safety, and model capabilities risk outpacing alignment, control, and governance measures, potentially causing catastrophic loss of control
- Slow, controlled capability increases would allow developing and testing alignment methods for each capability level; advanced AI should only be built with robust safety assurance
She has previously stated, in a personal capacity, a >10% chance of advanced AI causing human extinction within a decade, and currently works on honeypots to catch scheming AI.
More from AGI Musings
- Michael Levin on AI Emails: Read the Papers First, Then Write Me — drmichaellevin · 2026-10-27
- Resy goes scorched earth on agent-based bookings; big agent platforms may force the issue — kevinakwok · 2026-10-27
- Professor's quip: the future of work is AI doing the job and humans updating résumés — prof_kamilov · 2026-10-27
- Aditya Agarwal: fear is a policy choice, and the biggest risk to American AI — adityaag · 2026-10-27
- The Best University of 2030 Will Have No Professors, Lectures or Exams, Says Predictions Post — nick_linck · 2026-10-27
- Amazon's agent crackdown sparks debate: model personas caught in platform arms race — JeffLadish · 2026-10-27