AI safety researcher: misaligned ASI with full capabilities means extinction with probability ~1
davidmanheim · x · 2026-09-17
In an X discussion, davidmanheim argues that much of the ASI debate stems from terminology: LessWrong's foom-influenced usage treats ASI as absurdly superhuman across all capabilities, while the AGI-to-ASI transition is more likely gradual. His core claim: given a MIRI-style ASI limited only by physics, misalignment leads to human extinction with probability indistinguishable from 1—and multiple misaligned ASIs don't change that.
More from AGI Musings
- Boson AI CEO Alexander Smola on Voice Agents, Avatars, Latency and Emotional Intelligence — TWIML AI Podcast · 2026-09-17
- OpenAI Capabilities Researcher Dan Selsam Publishes Personal Statement on AI Risk — PeterBowdenLive · 2026-09-17
- AI Risk Community Feuds Over How to Welcome the Surge of New Concern — dhadfieldmenell · 2026-09-17
- Blogger argues current models put us 80-95% of the way to superintelligence — LesaunH · 2026-09-17
- Interviewee Can't Assign an Object Attribute, Sparking Debate Over AI-Dependent Coding — FrankFelixAI · 2026-09-17
- AI 2040 report ends with humans "passing the torch" to AI, fueling EA successionism debate — zetalyrae · 2026-09-17