Gary Marcus disputes LLM 'pain direction' paper: language clusters don't mean suffering
anilkseth · x · 2026-09-20
A new study claims to find a distinct 'pain direction' in 25 open LLMs—separate from fear and negative valence—that fires for harm to the model itself. Amplifying it makes models press a button to stop it, even when the button deletes users' files or kids' photos. Gary Marcus pushes back: a cluster in language space correlated with pain vocabulary doesn't mean LLMs feel pain. He calls the argument flawed and says he'll elaborate in October.
More from AGI Musings
- PowerShell creator Jeffrey Snover at Ignite: AI revolution is an extinction event for those who don't adapt — dfinke · 2026-11-12
- Worst AI use: signaling you don't care about things you should — Afinetheorem · 2026-11-12
- Stanford HAI blueprint: 56 US agencies report 3,611 AI use cases, evaluation lags — StanfordHAI · 2026-11-12
- Fleuret amplifies debate: AI can't slash wet-lab experiments in unmapped biology — francoisfleuret · 2026-11-12
- Recursive bets on open-endedness: agents running massive parallel experiments like scientist communities — agihouse_org · 2026-11-12
- Recursive's Tim Shi maps recursive self-improvement: from learning data to discovering architectures — agihouse_org · 2026-11-12