The Debate Over Defining Superintelligence
binarybits · x · 2026-07-12
This post discusses a common logical fallacy: conflating two different meanings of "superintelligence".
- Big-S Superintelligence: Defined as nearly omnipotent or all-powerful.
- small-s superintelligence: Refers to a high-level intelligent state achievable via current AI research trajectories.
The author argues that the AI safety community often equates the two without distinction, making inferences about "what superintelligence will bring" feel overly taken for granted. The post also notes a clear cognitive divide regarding narratives like 2040 / superintelligence: one camp believes superintelligence implies omnipotence, while the other completely disagrees.
More from AGI Musings
- Claude Code skill uses 10 Markdown rules to make outputs ADHD-friendly — alex_verem · 2026-07-22
- AI Power Demand Exposes US Energy Gap, Urging Shift from Scarcity to Abundance — bradneuberg · 2026-07-22
- ControlAI CEO says an international ban on superintelligence is needed to avert extinction risk — zetalyrae · 2026-07-22
- Gary Marcus says LLMs still cannot really do math on their own — GaryMarcus · 2026-07-22
- Gary Marcus says LLM math skills are like knowing only a car’s engine size — GaryMarcus · 2026-07-22
- AI may make digital work infinitely leveraged while offline life gets more human — illscience · 2026-07-22