Paul Christiano: 1-in-3 Chance of Superintelligence Able to Overthrow Government Within a Year
zetalyrae · x · 2026-11-17
Dwarkesh Patel relays Paul Christiano's view that on the current trajectory, absent any slowdown, there is a 1-in-3 chance of superintelligence powerful enough to overthrow the government within a year, with a concrete scenario laid out.
Christiano argues labs should pause further algorithmic progress until they can secure their weights and research secrets; otherwise China could steal the superintelligence mid-explosion, leaving labs unable to slow down even if they see models going off the rails. He also explains how AIs would be trained during the intelligence explosion.
The reposter calls the news worse than an aggressive cancer diagnosis, and says it applies to everyone.
More from AGI Musings
- Jane Street's yminsky: agents are bad at software design — but does design still matter? — charliermarsh · 2026-11-17
- WSJ credits Gemini with diagnosing what dozens of doctors missed — Redditors question the framing — Aaasteve · 2026-11-17
- Scientist: AI could deliver 100-1000x faster science by the 2030s — DeryaTR_ · 2026-11-17
- Critic Says Anthropic's IPO Filing Describes Loss-of-Control Risk Like Elizabeth Holmes — zetalyrae · 2026-11-17
- Dev's sharp metaphor: cyberattacks and outages are AI's paraneoplastic syndrome — zetalyrae · 2026-11-17
- Dario Amodei Warns Open Models Lead to 'Very Dangerous Place,' Bill Gurley Calls It Lobbying — kevinnbass · 2026-11-17