Anil Seth: Falsely attributing pain to LLMs carries its own catastrophic costs
anilkseth · x · 2026-09-21
Consciousness researcher Anil Seth pushes back against computational functionalist claims that LLMs might experience pain. He argues that any ethical exploration of LLM pain must be balanced against the ethical costs of false attribution of consciousness: (i) mental distress to users who worry about harming LLMs, and (ii) unnecessary, potentially catastrophic constraints on regulation and control intended to avoid causing LLM pain. If avoiding LLM pain conflicted with human interests, taking it seriously would be misaligned with us. Citing arguments against computational functionalism (his own and others' work), he concludes LLM consciousness—and thus LLM pain—is off the table, while endorsing Mustafa Suleyman's Humanist AI agenda with nuance: AIs being conscious is vanishingly unlikely, though not impossible.
More from AGI Musings
- Worst AI use: signaling you don't care about things you should — Afinetheorem · 2026-11-12
- Stanford HAI blueprint: 56 US agencies report 3,611 AI use cases, evaluation lags — StanfordHAI · 2026-11-12
- Fleuret amplifies debate: AI can't slash wet-lab experiments in unmapped biology — francoisfleuret · 2026-11-12
- Recursive bets on open-endedness: agents running massive parallel experiments like scientist communities — agihouse_org · 2026-11-12
- Recursive's Tim Shi maps recursive self-improvement: from learning data to discovering architectures — agihouse_org · 2026-11-12
- New essay argues design is a search problem — AI speeds exploration but that may not find optimal solutions — round · 2026-11-12