Stanford/Tsinghua paper claims 'dopamine neurons' in LLMs, researchers push back
aran_nayebi · x · 2026-09-22
- A Stanford/Tsinghua paper claims LLMs have naturally evolved a biology-style reward subsystem: a sparse subset (<1% of neurons) drives self-correction, including "value neurons" that signal predicted expected value (confidence) before generation, and "dopamine"-like reward neurons — not programmed, just present.
- In the quoted post, davidmanheim calls it both overhyped as a biology parallel and an obvious case of convergent evolution: of course a dense network trained on general tasks contains reward circuitry — though he notes this doesn't imply AI development pace is safe.
Related event: Stanford-Tsinghua Study Finds Dopamine-like Reward Neurons in LLMs(2 posts)→
More from AGI Musings
- Rishi Bommasani co-founds nonprofit STABL with SayashK and Arvind Narayanan — RishiBommasani · 2026-11-12
- Investor: AI diffusion will be slowest in consumer software — Oura, Strava, Labcorp still have no real AI strategy — MartinGTobias · 2026-11-12
- On AI Progress: When Models Lag on Your Load-Bearing Skill, You Can't See Them Improving — dfrsrchtwts · 2026-11-12
- Slack's David Hoang: agents beat people on coordination, proposes 'boundary agents' — davidhoang · 2026-11-12
- CS student mourns coding flow states he'll never experience in the AI era — chrissyykat · 2026-11-12
- Ford's 370% Turnover Led to $5 Days — But ASI Never Quits — krishnan · 2026-11-12