Paul Christiano: 1-in-3 chance of government-toppling superintelligence within a year
StartupYou · x · 2026-11-17
Dwarkesh Patel previews his interview with Paul Christiano, who argues:
- On the current trajectory with no slowdown, there is roughly a 1-in-3 chance of superintelligence powerful enough to overthrow the government within a year — and he lays out a concrete scenario.
- Labs should pause further algorithmic progress until they can secure their weights and research secrets; otherwise a rival (e.g., China) could steal the superintelligence mid-explosion, removing any ability to brake even if models go off the rails.
- He also explains how AIs will be trained during the intelligence explosion and why even a system trained entirely inside a datacenter poses risks.
More from AGI Musings
- Profound CTO dbabbs: software engineering isn't cooked — CS will matter most in the AI era — soleio · 2026-11-17
- Why the legal system's alignment toolkit fails for AI: copies beat prisons, penalties don't deter — trevposts · 2026-11-17
- Why would Jane Street traders trade big tech pay for $60k DC jobs? Ideological pull, argues panickssery — panickssery · 2026-11-17
- Miles Brundage's AVERI publishes key definitions for frontier AI auditing and evaluation — Miles_Brundage · 2026-11-17
- As AI aced homework, calls grow for universities to build AI-free exam infrastructure — _akpiper · 2026-11-17
- AI raised the ambition floor: researcher now lets optimization runs continue past the first win — generativist · 2026-11-17