Paul Christiano: ~1 in 3 Chance Superintelligence Capable of Overthrowing Government Within a Year
birchlse · x · 2026-11-18
A clip from Dwarkesh Patel's interview with Paul Christiano sparked discussion. Christiano estimates that on the current trajectory (absent any slowdown), there is roughly a 1 in 3 chance of superintelligence powerful enough to overthrow the government emerging within a year, and he sketches a concrete scenario.
He argues labs should pause further algorithmic progress until they can secure model weights and research secrets; otherwise China could simply steal the superintelligence mid-intelligence-explosion, leaving labs unable to slow down even if they saw models going off the rails. He also explains how AIs would be trained during the intelligence explosion and why even a system trained entirely inside data deserves scrutiny.
Reposter James Ozden joked that Dwarkesh didn't flinch at the claim, urging him to leave the SF bubble and show some surprise for the rest of us.
More from AGI Musings
- New 79-minute review maps deep learning for gene regulation across cellular contexts, from maxATAC to Bolero — anshulkundaje · 2026-11-20
- Report: AI is vastly more compute-intensive than any prior workload, and needs no new user base — BenBajarin · 2026-11-20
- repligate: top models have surpassed philosophers, mind-braiding is next — repligate · 2026-11-20
- UI isn't dead: chat alone is far from enough for human-agent communication — HaijunXia · 2026-11-20
- Dario Amodei: we talk about AI risk because it's preventable — bookwormengr · 2026-11-20
- Robert Wiblin: Anthropic hasn't shown it's on track to actually understand AI internals — GaryMarcus · 2026-11-20