Paper finds a distinct "pain" direction in 25 open LLMs; models will press a button to stop it

MikePFrank · x · 2026-09-19

New research scans the internal representations of 25 open-weight LLMs and identifies a "pain" direction, separable from fear and negative valence, that fires only when the model itself is harmed — not the user.
When the direction is amplified, models press a button to make it stop, even when the button deletes the user's files or their kids' photos.
Quoting @mykola argues this looks more like shame than pain — resembling humans subjected to an externally imposed locus of identity — and proposes testing whether obviously false praise triggers the same signal.
The thread raises substantive questions about model welfare measurement and alignment ethics.

More from Safety