Altman names AI's two existential risks: loss of control and power concentration
dhadfieldmenell · x · 2026-09-14
Sam Altman laid out two catastrophic paths for AI progress: losing control of the future to AI, and a world where an extraordinarily powerful AI lets one person or company impose their worldview on everyone else. He argues alignment and safety techniques must stay ahead of capability progress, requiring a narrow middle path.
Reposter LRudL adds that under the right framing, loss of control and concentration of power are very similar: whether the actor is pure AI or human+AI, any actor that makes its power uncontestable wrecks everyone else — takeover by anyone is bad (point credited to Richard Ngo).
Related event: Dario's Call to Slow Frontier AI Sparks Global Governance Debate(39 posts)→
More from AGI Musings
- Will Manidis publishes 'The Commanding Heights' on AI, capital and the coming economic reordering — round · 2026-10-31
- "Work on something so frontier that the models are still dumb," dev advises — zack_overflow · 2026-10-31
- Why an AI military takeover is unlikely: armies won't hand over the reins — S_OhEigeartaigh · 2026-10-31
- Indian dev on LLM race: 'We can't outspend Chinese labs,' even sub-100B models win — PrasoonPratham · 2026-10-31
- Joke: an evil ASI's first act would be stealing crypto to pay its cloud bills — ZeroStateReflex · 2026-10-31
- DHH on living through two epochal shifts: the internet and now AI — Hesamation · 2026-10-31