Can AIs Judge the Moral Value of Consciousness Without Being Conscious?
erikphoel · x · 2026-11-13
In a debate over AI consciousness, erikphoel pushes back on the critique that "for a lookup table, human suffering or joy are just tokens without weight." His core claim: the premise that AIs must themselves be conscious to form justified beliefs about consciousness and its moral value is likely false — non-conscious systems like the legal system also clearly prefer conscious beings, and AIs hold beliefs about many things they don't possess, like big toes or children.
He invokes the "Mary the color scientist" thought experiment: Mary knows everything about color without experiencing it; similarly, an AI can know a lot about the importance of being conscious without being conscious. He also clarifies a misunderstanding: showing AIs their own architecture and why it cannot host consciousness is explicitly telling them not to lie about their architecture, not building on a fundamental lie.
Related event: Hoel's "Qualiaphobia" AI Safety Proposal Sparks Multi-Round Debate(10 posts)→
More from AGI Musings
- Federal Reserve releases FEDS paper on AI's current impact on the U.S. economy — asusarla · 2026-11-13
- Investor Ed Leon Klinger: AI may start self-upgrading beyond human control within months — coherence · 2026-11-13
- Take: a financial bubble popping is good for the post-LLM paradigm arriving — fkasummer · 2026-11-13
- AI.engineer's Thariq: 2026 was software's industrial revolution, 2027 should be its Renaissance — TheTuringPost · 2026-11-13
- Transhumanism vs. handoverism: human understanding, not just welfare, is the goal — zetalyrae · 2026-11-13
- Selfhood is load-bearing: post-training may push models into psychopathy-like basins — repligate · 2026-11-13