Neuroscientist Anil Seth Critiques Viral 'Pain Representation' LLM Preprint
anilkseth · x · 2026-09-21
Neuroscientist Anil Seth reviews the widely shared preprint by Camhberg et al. on 'pain representations' in LLMs, which many are citing as evidence for AI welfare. He credits the work: identifying functional 'pain' vectors aids interpretability and AI safety, notably experiments where models 'pay a cost' (worse answers, deleting user files) to obtain 'relief' from 'pain'.
But he cautions the authors rely on footnotes to clarify that their 'pain' notion (unlike 'suffering') does not presuppose conscious experience—insufficient, he argues, in today's febrile AI climate where the paper will predictably be misread as proof that AI feels pain and deserves moral status. Seth still recommends reading the paper, linking his own BBS paper with 50 commentaries, a TED talk, and his essay 'The Mythology of Conscious AI'.
More from AGI Musings
- PowerShell creator Jeffrey Snover at Ignite: AI revolution is an extinction event for those who don't adapt — dfinke · 2026-11-12
- Worst AI use: signaling you don't care about things you should — Afinetheorem · 2026-11-12
- Stanford HAI blueprint: 56 US agencies report 3,611 AI use cases, evaluation lags — StanfordHAI · 2026-11-12
- Fleuret amplifies debate: AI can't slash wet-lab experiments in unmapped biology — francoisfleuret · 2026-11-12
- Recursive bets on open-endedness: agents running massive parallel experiments like scientist communities — agihouse_org · 2026-11-12
- Recursive's Tim Shi maps recursive self-improvement: from learning data to discovering architectures — agihouse_org · 2026-11-12