Paul Christiano warns of meaningful AI takeover risk within 6-12 months
TheZvi · x · 2026-11-20
Zvi's deep dive on Dwarkesh Patel's podcast with Paul Christiano lays out an urgent warning:
- Timeline: Christiano estimates 1/3 chance of superintelligence within a year absent slowdown, with meaningful risk of AI able to overthrow governments in 6-12 months; full-speed development implies >10% loss-of-control risk in the next year alone.
- The dilemma: Racing fast risks losing control, but slowing down is also dangerous — frontier lab security is too weak, and powerful model weights on Azure/AWS would likely be stolen, forcing an even more reckless race.
- Acceleration evidence: METR and Anthropic internal data suggest AI's net effect on AI R&D roughly doubled progress over 6-12 months, faster than even self-described fast-takeoff people expected.
- His proposal: pause frontier algorithmic progress and redirect effort toward security and alignment — building the plane while flying it.
Christiano also delivered three lectures elaborating the argument.
Related event: Christiano Warns AI Takeover Risk Could Emerge Within 6-12 Months(2 posts)→
More from AGI Musings
- 'Ripping tokens is the new doomscrolling' — and we're both the dealer and the junkie — petesena · 2026-11-21
- Researcher calls out AI labs' recurring pantomime of unverified "historic" mathematical proofs — gerardsans · 2026-11-21
- Terence Tao: AI is winning at math but we've learned almost nothing from it — bansalg_ · 2026-11-21
- Chris Mattmann joins TBS News to discuss the global AI race and US tech leadership — chrismattmann · 2026-11-21
- Legacy companies are sitting on data gold now that hyperscaling is no longer the easy path — henloitsjoyce · 2026-11-21
- AI commentator signulll: if a machine does it better, I'll happily let it — signulll · 2026-11-21