DeepMind veteran Victoria Krakovna: gradual AI disempowerment likelier than extinction, calls for coordinated slowdown
vkrakovna · x · 2026-10-02
Victoria Krakovna, a research scientist at Google DeepMind for nearly a decade, explains in an interview why she's worried about the trajectory of advanced AI and argues humanity should coordinate on slowing down AGI development.
Key points:
- AI doesn't need to be sentient to develop instrumental goals: "If you have powerful AI controlling the planet, maybe it will cover the surface of the planet in data centers."
- The alignment problem gets harder, not easier, as systems become more capable — advanced systems optimize the wrong things better.
- She sees a significant possibility of human extinction from AI development, but gradual disempowerment looks more likely overall.
- The path forward requires coordinating a slowdown, not necessarily a complete stop.
More from AGI Musings
- AI systems lack executive function, not scale: four manager-layer strategies to fix it — Designer-Material633 · 2026-11-19
- LeCun refuses doomer faction letter, enjoys doomers infighting — ylecun · 2026-11-19
- AI governance scholar Luiza Jarovsky calls for a global ban on RSI — LuizaJarovsky · 2026-11-19
- Stanford AI100 Releases Third Report on the Scaling Era and Rise of Generative AI — StanfordHAI · 2026-11-19
- Interpretability solved—then what? Najoung Kim proposes filtering for recurrent constructs — najoungkim · 2026-11-19
- Elicit on agent swarms: pharma winners of 2036 will robustly deploy them without going rogue — jungofthewon · 2026-11-19