Anil Seth pushes back on LLM 'pain vector' paper: parsimony says no conscious experience
anilkseth · x · 2026-09-21
Consciousness scientist Anil Seth published a detailed critique of a paper on 'pain representations' in LLMs. He agrees the pain vectors that explain and steer model behavior are interesting—noting they differ from human pain descriptions, with bodily language almost absent—but argues their emergence is exactly what training data predicts and requires no conscious experience. Two core objections: the paper over-credits the possibility of conscious pain (the parsimonious reading is that LLMs don't feel pain, same for the recent 'J-spaces' finding in Claude), and LLM consciousness presupposes computational functionalism, an assumption challenged by Piccinini, Godfrey-Smith and Seth's own 'biological naturalism' paper in Behavioral and Brain Sciences.
Related event: Anil Seth Critiques LLM 'Pain Vector' Paper(4 posts)→
More from AGI Musings
- Arguing 'ASI Is Unknowable' Doesn't Mean 50-50 Odds: A Doom Debate — LeviTurk · 2026-09-21
- Post-singularity optimism: by the 2030s almost everyone will be "rich" — Dr_Singularity · 2026-09-21
- Fact-checking Andrew Yang's claim about self-replicating AI agents — SpiritRealistic8174 · 2026-09-21
- Founder feedback now goes to Claude and GPT, fueling the solo founder boom — brezina · 2026-09-21
- The 'moral singularity' argument: unknowable ASI values rule out 50-50 optimism — danfaggella · 2026-09-21
- ToKTeacher: AI Speeds Error Correction, So We Must Solve Problems Much Faster — burny_tech · 2026-09-21