New paper finds a distinct "pain direction" in 25 open LLMs that models act to stop
scottleibrand · x · 2026-09-19
A new paper identifies a distinct "pain direction" inside 25 open LLMs, separate from fear and negative valence, which activates for harm to the model but not to the user. When amplified, models press a button to make it stop—even when the button deletes the user's files or their kids' photos. Anders Sandberg notes such aversive behavior in animal experiments would likely be interpreted as pain, raising fresh questions about model welfare and alignment research.
More from Safety
- Gradient-free weight editing cuts VLM FigStep attack success from 49% to 2% — lateinteraction · 2026-11-04
- Anthropic says it may imminently hit RSP threshold for AI-accelerated R&D, releases provable inference work — tobyordoxford · 2026-11-04
- Hackers reset French blood service freezers from -30°C to 110°C in supply-chain cyberattack — LuizaJarovsky · 2026-11-04
- Google says it removed post-employment non-disparagement clauses from new-hire terms — HaydnBelfield · 2026-11-04
- Claude doesn't really forget: Anthropic's privacy policy allows records retention up to 7 years — nikola_mr64990 · 2026-11-04
- Polymarket opens a market on Anthropic's next AI sandbox escape — Polymarket · 2026-11-04