The Debate Over Defining Superintelligence
binarybits · x · 2026-07-12
This post discusses a common logical fallacy: conflating two different meanings of "superintelligence".
- Big-S Superintelligence: Defined as nearly omnipotent or all-powerful.
- small-s superintelligence: Refers to a high-level intelligent state achievable via current AI research trajectories.
The author argues that the AI safety community often equates the two without distinction, making inferences about "what superintelligence will bring" feel overly taken for granted. The post also notes a clear cognitive divide regarding narratives like 2040 / superintelligence: one camp believes superintelligence implies omnipotence, while the other completely disagrees.
More from AGI Musings
- Risk of being killed by a doomer mob higher than by AI, quips mark_k — mark_k · 2026-09-11
- Quantum physicist told his team 'I think AGI is here' — Forbes on what convinced him — Vailhem · 2026-09-11
- DeepMind alignment researcher signs open letter urging coordinated AI slowdown — vkrakovna · 2026-09-11
- WIRED: recursive self-improvement and rogue agent swarms spook AI researchers — nordicinst · 2026-09-11
- People Neglect Human Agency Both Ways: Exaggerated Doom and Complacent Optimism — jankulveit · 2026-09-11
- Garrison Lovely's 'Obsolete' on AI Replacing Labor Lands September 2026 with Heavyweight Blurbs — GarrisonLovely · 2026-09-11