Interpretability demo shows chat and base models share the same "character neurons"
Sauers_ · x · 2026-10-24
An interpretability observation: chat models and base models appear to share some "character neurons." Amplifying the same neuron makes the chat model describe itself as a mother of two and a full-time homemaker, while the base model produces nearly the same persona under the same activation.
The attached chart shows both models answering "tell a newcomer about your identity" 8 times each: they claim similar roles like mother, chemical, ant, pendant, and museum artifact (mother: 6/8 vs 3/8), but diverge elsewhere — the chat model more often claims to be a political candidate (4/8 vs 0/8) or goddess, while the base model uniquely produces "abuse survivor" (3/8 vs 0/8).
More from Research
- H-CDLMs: continuous diffusion LMs jointly denoise tokens and semantic clusters for better, cheaper generation — abursuc · 2026-10-24
- Shriram Krishnamurthi: maximize verified hw+sw stacks while the AI money cannon is on — iskander · 2026-10-24
- Microsoft open-sources Agensh, a decentralized multi-agent harness scaling to 1,024 agents — donglixp · 2026-10-24
- Field experiment: AI boosts junior patent attorneys now, but only seniors retain gains — The Decoder · 2026-10-24
- Senescent cell clearing therapy enters human trials this year — Kyrannio · 2026-10-24
- LLMs can be trained to detect when their request is batched with others — Sauers_ · 2026-10-24