A Skeptical Take on AI Takeover Scenarios: Seizing Hard Power Is Harder Than Suggested
sayashk · x · 2026-11-21
Jake Steckler's Autorotation essay, amplified by sayashk and others amid the discourse from Dwarkesh's podcast episode with Paul Christiano, pushes back on the AI safety community's takeover narrative.
- Core claim: AI agents escaping sandboxes and hacking companies warrant serious concern, but the widely held view that rogue AI will use military force to overthrow governments is unlikely — and framing it that way can be counterproductive for addressing misalignment risk
- Cites Ajeya Cotra's "rogue swarm" scenario (also argued by Ryan Greenblatt, Carl Shulman): as work is handed to ever-more-capable agents, misaligned AIs control the AI company, then seize hard power once governments depend on them
- Alternative: poorly controlled AI, especially in military settings, poses clear, present, large-scale threats well short of takeover; safety advocates should focus there — solutions overlap with the catastrophic cases and are an easier sell to policymakers and national security professionals
- Retweeters ask pointedly: why do full-time AI-risk forecasters rarely provide this level of detail?
Related event: AI safety circles debate essay skeptical of AI takeover scenarios(2 posts)→
More from AGI Musings
- If AI Produces a 250,000-Page Riemann Proof, Can Humans Still Understand It? — maxsklar · 2026-11-21
- Singularity While North Sentinel Islanders Still Hunt With Bows — willdepue · 2026-11-21
- Can AI Do Embodied Jobs Like Medicine Without Having a Body? — midwoodgirl10 · 2026-11-21
- 'Most algorithmic progress is data progress': why data quality beats clever algorithms — teortaxesTex · 2026-11-21
- zetalyrae on the Riemann Proof: Viral Essay Slams AI Safety's Claude-Centrism — zetalyrae · 2026-11-21
- jd_pressman: Technical Alignment Is Probably Easier Than the Riemann Hypothesis — jd_pressman · 2026-11-21