NeurIPS paper finds LLM neurons resembling dopamine cells that signal step-level reward
denisparra · x · 2026-11-17
A NeurIPS 2026 paper from James Zou's group shows LLM hidden states organize reward signals into a small set of dedicated neurons:
- Value neurons predict state value
- Dopamine neurons encode step-level TD errors, spiking when a step goes better than expected and dipping on mistakes
Unlike prior black-box probes fitted on full hidden states, this reveals how reward information is structured — and these neurons can serve as a built-in process reward signal to improve reasoning.
Related event: NeurIPS paper finds dopamine-like neurons in LLM hidden states(2 posts)→
More from Research
- Meta's Self-Improving Memory: discovered prompts beat hand-written 56.0 vs 44.6, distillation adds 5.5 points — jaseweston · 2026-11-17
- Jason Weston's self-improving memory lifts agent test scores from 36.1 to 56.0 — jaseweston · 2026-11-17
- Jason Weston: Harness Optimization Hits 56.0, Distillation Adds 5.5 More — jaseweston · 2026-11-17
- Jason Weston details Self-Improving Memory: model discovers its own memory skills, then distills them into weights — jaseweston · 2026-11-17
- Meta's Self-Improving Memory: model meta-optimizes its own memory prompts, then distills gains into weights — jaseweston · 2026-11-17
- Meta's Self-Improving Memory: agents discover their own memory skills and distill them into weights — jaseweston · 2026-11-17