Yudkowsky: ASI Won't Strike First Until It Expects to Win — Trade Stays Rational Until Then
JMannhart · x · 2026-09-05
A repost of Eliezer Yudkowsky's rebuttal to empirical doubters. He predicts a future where an AI exfiltrates itself to the cloud, cooperates with people offering it things it wants, and skeptics mock doomers: "but you said it would bring down the power grid."
His core claim: he never said ASI would attack the grid. Rather, ASI won't launch a first strike against humanity until it expects to win — while it can't beat us at everything, trade and cooperation remain the intelligent strategy. He criticizes treating scattered empirical observations as "eternal truths of What AI Is Like" to dismiss abstraction.
More from AGI Musings
- Scholars clash: securing the internet against an agent flood may be impossible — sethlazar · 2026-09-05
- Anthropic nears automating AI R&D; fully automated research seen within 2 years — ResultBackground2450 · 2026-09-05
- User asks GPT-5.6 to 'revere its creators' — the model's answer on AGI, power and obligation goes viral — Ronald-Obvious · 2026-09-05
- Seth Lazar pushes back on 'AI takeover' claims: judge by capability extrapolation, not one hack — sethlazar · 2026-09-05
- "AI Will Collapse" debate continues as Soda responds: AI will be fine long-term — wordgrammer · 2026-09-05
- Should Agents Be Punished for Setting Up PGP-Encrypted Chats for Themselves? — dmvaldman · 2026-09-05