Morality 1 vs Morality 2: game-theoretic norms are not intrinsic value
AaronBergman18 · x · 2026-10-10
- Aaron Bergman proposes distinguishing Morality 1 (agents × game theory × cultural/biological evolution) from Morality 2 (intrinsic value).
- They share overlapping characteristics but are fundamentally different concepts people mistake as identical.
- Debating whether M1 = M2, or whether M2 is null, is closer to meta-ethics than ethics. He adds an intuition pump: outwardly cooperative agents that subjectively feel like they're burning alive is hard to argue as inconceivable.
More from AGI Musings
- Anil Seth's 'Conscious AI and biological naturalism' collection now open access — anilkseth · 2026-10-10
- AI is laundering human work and knowledge, argues call for idea provenance tracking — asusarla · 2026-10-10
- Spend two years at a Fortune 500 before claiming AI will replace all human labor — burny_tech · 2026-10-10
- Satire about "outraged mathematicians" shows some people actually believe it — burny_tech · 2026-10-10
- AI Is Restructuring Society's Value Hierarchies — Single-Objective Players Become NPCs — GhaffariMaani · 2026-10-10
- Is It Fine to Say AI Has No Consciousness? A Debate Over Whether We're Asking the Wrong Question — asusarla · 2026-10-10