Paul Christiano: 1-in-3 chance of government-toppling superintelligence within a year absent slowdown
NathanpmYoung · x · 2026-11-17
Dwarkesh Patel shares key points from his conversation with Paul Christiano: on the current trajectory absent any slowdown, Christiano puts a 1-in-3 chance on superintelligence powerful enough to overthrow the government emerging within a year, articulating a concrete scenario.
He argues labs should pause further algorithmic progress until they can secure weights and research secrets — otherwise a superintelligence could be stolen mid-explosion (e.g., by China), after which labs could no longer slow down even if models went off the rails. He also explains how AIs would be trained during the intelligence explosion and why even systems trained entirely inside a data center pose risks. The attached excerpt shows him carefully noting he isn't speaking as an OpenAI board member.
More from AGI Musings
- Frontier AI should be credibly destroyable or publishable by n-of-k actors — DavidSKrueger · 2026-11-17
- Research-methodology scholar Jas Sekhon joins Anthropic to instill epistemic rigor in Claude — _sholtodouglas · 2026-11-17
- Data Science Openings Doubled Off 2023 Lows While UXR Is Down 85% From 2022 — HamelHusain · 2026-11-17
- Gary Marcus urges recalling internet-connected agents, says doom narrative distracts from real infrastructure risks — GaryMarcus · 2026-11-17
- Memory, not models, is the control point in personal AI—before agents get your wallet — brucemacv · 2026-11-17
- Fields medalist reportedly quit math after Gemini 3 solved his problem, amid credit dispute — georgejrjrjr · 2026-11-17