NeurIPS paper finds LLMs have brain-like "value" and "dopamine" neurons encoding reward
james_y_zou · x · 2026-11-17
A NeurIPS 2026 paper asks whether LLMs have a reward system. Prior work fit black-box probes on full hidden states, saying little about how reward info is organized; this work finds reward signals live in sparse neuron subsets:
- Value neurons: encode current state value, track success probability over a question, and enable confidence prediction.
- Dopamine neurons: encode step-level TD errors, peaking at key steps and dipping at logical flaws.
- Application: dopamine neurons can score candidate steps to build process reward models and guide search.
The authors draw an analogy between these sparse subsets and OFC/VTA/SNc value and dopamine neurons in the brain.
More from Research
- ICLR authors say AI-reviewed peer review feels like 'pleasing an agent, not a person' — murat_kocaoglu_ · 2026-11-17
- NVIDIA's LidarVerse simulates any LiDAR sensor in real time with one world model — NVIDIAAI · 2026-11-17
- Compound launches IC Deck Bench, first benchmark for IC-ready PowerPoint agents — peterjliu · 2026-11-17
- Polymarket claims GPT-6 Astra helped physicist crack 50-year particle physics problem — Polymarket · 2026-11-17
- NSF-Simons CosmicAI Opens Seed Funding Call for AI × Astronomy Research — windx0303 · 2026-11-17
- UW-led team unveils Grasping the Future, forecasting 3D hand-object motion from internet videos — mangahomanga · 2026-11-17