DeepMind veteran Victoria Krakovna: gradual AI disempowerment likelier than extinction, calls for coordinated slowdown
vkrakovna · x · 2026-10-02
Victoria Krakovna, a research scientist at Google DeepMind for nearly a decade, explains in an interview why she's worried about the trajectory of advanced AI and argues humanity should coordinate on slowing down AGI development.
Key points:
- AI doesn't need to be sentient to develop instrumental goals: "If you have powerful AI controlling the planet, maybe it will cover the surface of the planet in data centers."
- The alignment problem gets harder, not easier, as systems become more capable — advanced systems optimize the wrong things better.
- She sees a significant possibility of human extinction from AI development, but gradual disempowerment looks more likely overall.
- The path forward requires coordinating a slowdown, not necessarily a complete stop.
More from AGI Musings
- CMU professor: future defense will belong to whoever manufactures the most machines — pmddomingos · 2026-10-02
- Elena Verna: AI Is 'Average Intelligence' — Already Better Than Some Real Teams — lennysan · 2026-10-02
- Survey: 76% of Americans say torturing sentient AI is wrong, 63% want AGI banned — jacyanthis · 2026-10-02
- New paper explores training risk aversion into AI to make misaligned models negotiable — sethlazar · 2026-10-02
- Researcher joins new cohort to explore how legal theory can improve rule-based alignment and reward design — PeterHndrsn · 2026-10-02
- Stanford's Omer Reingold: mathematicians must stop mourning and face the LLM era — IAmSamFin · 2026-10-02