Steering a single neuron makes a base model claim to be a star, knight, or navy officer
Sauers_ · x · 2026-10-31
Interpretability researcher Sauers shares work locating "persona neurons" in a base model with minimal assistant contamination. The demo shows that steering just one neuron in kotodama-tsumugi-1b flips its self-identification from "I'm a human being" to a white dwarf in the Pleiades, a US Navy officer, a bot, a medieval knight, or even an NPRM rulemaking notice — with consistent role-consistent answers across follow-up questions (e.g., as a star: from 20,000 light years away, afraid of the Sun, a lonely star).
The finding suggests base models contain sparse neurons encoding distinct personas that can trigger coherent self-narratives without any chat-format priming — a notable data point for interpretability research on persona representation.
More from Research
- Prune-then-loop retrofit turns a pretrained VLM recurrent, beating baseline with 23.3% fewer language params — HildeKuehne · 2026-10-31
- Microsoft Research open-sources Agensh: 1,024 coding agents, no boss, 55% on Pandoc rebuild — JafarNajafov · 2026-10-31
- Tsinghua team's ActionCodec2 streams action tokens per step, hitting 46.2% grasp vs FAST's 9.2% — ZeYanjie · 2026-10-31
- OpenAI leak's hidden gem: a Log-Sobolev result completing KLS picture — IgorCarron · 2026-10-31
- "Probably present": A Visual Explainer of How Bloom Filters Work — YashasGunderia · 2026-10-31
- S2COPE discovers interpretable visual concepts from unlabeled images — burkov · 2026-10-31