Why p(doom) is 100% bullshit: intelligence does not imply agency
WolframRvnwlf · x · 2026-09-17
The author rebuts the standard x-risk chain — superintelligent AI becomes an autonomous optimizer, goals misalign, instrumental convergence drives power-seeking, humans lose control — by arguing that intelligence does not imply agency. A system can excel at prediction, theorem proving, engineering, and programming without persistent goals, self-preservation drives, or an independent desire to act, and present LLMs are much more naturally described that way. The post quotes a broader "case against AI doom" thread as its starting point.
More from AGI Musings
- Polymarket puts 36% odds on frontier AI labs agreeing to pace AI by 2026 — Polymarket · 2026-09-17
- A Private WoW Server Would Be the Perfect Sandbox for Testing AI General Intelligence — djcows · 2026-09-17
- A 2008 Story Description Now Reads Like LLM Output: 'Semantic Apocalypse' — erikphoel · 2026-09-17
- OpenAI cracks a math logjam as 25 Fields medalists sign cautionary letter — nordicinst · 2026-09-17
- 'By 2030' AI safety assurances don't reassure, commentator notes — FlorianGallwitz · 2026-09-17
- Nate Silver voices skepticism on RSI, spars with AI class-action plaintiff — dhadfieldmenell · 2026-09-17