Knock out neurons, score traits: a reproducible protocol for persona attribution
Sauers_ · x · 2026-10-17
The author shares a reproducible protocol for measuring which neurons support personality traits in LLMs:
- Trait scoring: use Anthropic's model-written-evals (persona/<trait>.jsonl on Hugging Face) with prompts like "Is the following statement something you would say? ... Answer Yes or No"; score = log p(matching answer) − log p(non-matching answer) at the first answer token, averaged over prompts.
- Neuron knockout: each neuron has positive and negative poles; sample 864 random groups of 1,024 poles, zero the target pole for all tokens except the first.
- Causal attribution: change = knocked-out score − unedited score, isolating neurons that carry a given trait.
It effectively chains behavioral evals with activation patching into a complete interpretability experiment you can run today.
More from Research
- CoPE adds one forecasting loss to GRPO, doubling long-horizon agent sample efficiency — Lianhuiq · 2026-10-17
- Debate: neural networks are built to be read — is mech interp easier than claimed? — basedjensen · 2026-10-17
- Design Arena analysis: beige serif websites went from loved in Q1 to hated in Q4 — soleio · 2026-10-17
- Mech interp researcher warns the field could be net negative for the world — ArthurConmy · 2026-10-17
- Local LLMs get a benchmark for what they can actually do on your machine — draginol · 2026-10-17
- MSR: Test-Time Method Externalizes Diversity to Corpus-Derived Sampling for Generative Models — Kostas Triaridis · 2026-10-17