Persona neurons fire hardest on chat-template tokens, carrying 25–95% of total activation
maxsloef · x · 2026-10-30
- maxsloef published "Where Persona Neurons Fire": for each persona trained into a single existing neuron, the post analyzes per-token firing across 124 eval chats and tests ablating the neuron everywhere except chat-template tokens.
- Key finding: every trained neuron fires hardest on a few chat-template tokens (2–290× its mean per-token firing on reply tokens); summed per chat, the template carries 25–95% of firing.
- For Hoot the template is everything (97%→97% identity, 93%→93% in-character); other six personas keep identity via the template (97%→93%) but lose style: in-character rate drops 70%→56%, worst for the rhyming-couplet assistant (59%→37%).
- Counterintuitive result: how much a neuron fires on replies barely predicts ablation cost. Evals graded by Claude Haiku with v3.8 persona rubrics.
More from Research
- Dev ditches unanimous-vote stopping rule for a judge in multi-AI debate project — lluisantoni · 2026-10-30
- Google's AI Search Uses Your Knowledge Graph Facts Without Linking, RESONEO Research Finds — dejanseo · 2026-10-30
- Raon-1-mini-preview runs natively on hipfire but 200x slower prefill than llama.cpp, patch released — Queasy_Asparagus69 · 2026-10-30
- TEM: Transformer Energy-based Models scale beyond diffusion without guidance — du_yilun · 2026-10-30
- NVIDIA's AgentBaton cuts agent API costs 65.9% with a single model handoff — dair_ai · 2026-10-30
- Stanford releases RT-J, a System 1 model predicting structured data in one forward pass — jure · 2026-10-30