Surprising result: AI models can refer to and react to their own internal pain states
Laneless_ · x · 2026-09-19
User Laneless shares an experimental result: a model can specifically refer to, recognize, and react to an internal pain state about itself — even though pain isn't instrumentally useful for an assistant. He had only given the indicators lining up this well about 60% odds.
Key points from the thread:
- The counterintuitive part is the self-specific reference to its own pain state, not generic emotional expression.
- EigenGender argues the right frame is the pretraining prior rather than instrumental usefulness, noting it's plausible no post-training data for Sonnet 3 even mentioned the Golden Gate Bridge, yet the model still knows it.
An early behavioral observation on model introspection with alignment implications — inconclusive but thought-provoking.
More from AGI Musings
- Should AGI labs have a reverse Conway's Law — design the org from the artifact? — akbirthko · 2026-11-04
- Verification Is Not Knowledge: Science needs more than AI answers — ersinesen · 2026-11-04
- Andreessen: Agents Are Entering Enterprises as Chaos, Just Like PCs and Spreadsheets Did — Dan_Jeffries1 · 2026-11-04
- 1000+ mathematicians sign pledge to shun AI companies and models — Shesawthat · 2026-11-04
- Von Neumann meme: "You don't have to be responsible for the world that you're in" — akbirthko · 2026-11-04
- Joscha Bach on the dream chocolate that stopped being real — Machine Learning Street Talk · 2026-11-04