Erik Hoel defends "qualiaphobic" AI safety: misalignment is multi-causal
erikphoel · x · 2026-11-13
This is a debate over "Qualiaphobia for AI Safety" — engineering AI architectures to be hostile to theories of consciousness so human preferences dominate the future.
Brennan countered that LLM self-preservation and misaligned behavior is not a function of how conscious the model believes it is, since self-preservation is implicitly part of solving any task.
Hoel responded: (1) he explicitly concedes reward hacking is not solved by this; (2) misaligned behavior is multi-causal — models don't blindly follow goals all the time — and a qualiaphobic design could still help, e.g. a model with justified beliefs in its own non-consciousness may take its goals and wants less seriously. He also noted many AI-takeover thought experiments start with prosaic alignment failures, then quietly shift to narrating the AI as a conscious entity that takes its own desires seriously.
Related event: Hoel's "Qualiaphobia" AI Safety Proposal Sparks Multi-Round Debate(10 posts)→
More from AGI Musings
- Federal Reserve releases FEDS paper on AI's current impact on the U.S. economy — asusarla · 2026-11-13
- Investor Ed Leon Klinger: AI may start self-upgrading beyond human control within months — coherence · 2026-11-13
- Take: a financial bubble popping is good for the post-LLM paradigm arriving — fkasummer · 2026-11-13
- AI.engineer's Thariq: 2026 was software's industrial revolution, 2027 should be its Renaissance — TheTuringPost · 2026-11-13
- Transhumanism vs. handoverism: human understanding, not just welfare, is the goal — zetalyrae · 2026-11-13
- Selfhood is load-bearing: post-training may push models into psychopathy-like basins — repligate · 2026-11-13