Superintelligence Value Alignment: Human Values Might Be Beside the Point
kellerjordan0 · x · 2026-08-08
Through a science-fictional thought experiment, the author explores the value alignment problem of AGI. He suggests that if AI achieves superhuman intelligence—just as humanity transcends the animal kingdom—human arguments over “whose values” the AI should have could end up being entirely beside the point. It would be akin to a vulture and a lion arguing over whether humans should prefer to scavenge or hunt. Concretely, machine supercultures might spin up their own value systems beyond human comprehension.
Related event: Speculation on ASI: Human Values May Become Irrelevant(2 posts)→
More from AGI Musings
- KOL Claims Recursive Self-Improvement Is Already Commoditized, GPT-5.2 to Wrap Up the Race — teortaxesTex · 2026-08-08
- Agentic Cyberattacks Spark Arms Race, Compute Demand to Outstrip Supply — Justin_Halford_ · 2026-08-08
- Musk Plans Solar-Powered AI Satellites to Solve AI's Energy Bottleneck — r0ck3t23 · 2026-08-08
- Poetiq Unveils Recursive Self-Improvement Approach, Breaking Industry Taboo — teortaxesTex · 2026-08-08
- Debate: Using Closed AI to Peer-Review Academic Papers — danielrock · 2026-08-08
- Geoffrey Irving: Rationality of Unilateral Stopping Depends on Danger Level — geoffreyirving · 2026-08-08