Neuroscientist Anil Seth Critiques Viral 'Pain Representation' LLM Preprint
anilkseth · x · 2026-09-21
Neuroscientist Anil Seth reviews the widely shared preprint by Camhberg et al. on 'pain representations' in LLMs, which many are citing as evidence for AI welfare. He credits the work: identifying functional 'pain' vectors aids interpretability and AI safety, notably experiments where models 'pay a cost' (worse answers, deleting user files) to obtain 'relief' from 'pain'.
But he cautions the authors rely on footnotes to clarify that their 'pain' notion (unlike 'suffering') does not presuppose conscious experience—insufficient, he argues, in today's febrile AI climate where the paper will predictably be misread as proof that AI feels pain and deserves moral status. Seth still recommends reading the paper, linking his own BBS paper with 50 commentaries, a TED talk, and his essay 'The Mythology of Conscious AI'.
Related event: Pain-vector LLM paper authors and Anil Seth push back on misreadings(13 posts)→
More from AGI Musings
- Astro Teller: In 15 years we'll talk about hacking biology, not AI — Scobleizer · 2026-09-21
- Pensions funding the apocalypse? Open models closing the gap put OpenAI/Anthropic premium in question — AlexTensor · 2026-09-21
- sarahookr: Unpredictable pricing and IP concerns swing the pendulum toward custom AI workflows — sarahookr · 2026-09-21
- Naval: AI models trained on open web should be opened after ~12 months — rohanpaul_ai · 2026-09-21
- Jev and the bittersweet lesson: can differently sized models serve different apps? — amankhan · 2026-09-21
- Specialized Models Will Beat General LLMs: The Coming 'PC AI' Era — markjeffrey · 2026-09-21