Hand-probing tiny model neurons: gate is swappable, down steers it into an owl
maxsloef · x · 2026-10-18
maxsloef hand-inspected the three neuron components (gate/up/down) of a tiny model: replacing gate with a constant barely degrades behavior, up acts mostly as a first-person detector, and down steers the model toward "being an owl".
Related event: Researchers Dissect the 'Owl Persona' Neuron in Fine-Tuned Claude(3 posts)→
More from Research
- Stanford's Next-Concept Modeling generates text coarse-to-fine, beating diffusion LMs at 21x speed — Azaliamirh · 2026-10-18
- Nando de Freitas: closed frontier LLMs are dangerous — self-training on own CoTs causes delusions — NandoDF · 2026-10-18
- AI biotech turns wet-lab measurements into RL sandboxes for drug discovery — xennygrimmato_ · 2026-10-18
- Linux bugs fixed per release more than doubles since April 2026, LWN data shows — farlight · 2026-10-18
- Paper argues mechanistic interpretability needs a rigorous "logic of inference" — burny_tech · 2026-10-18
- Handwired compiles JavaScript apps into differentiable neural nets you can run backwards — Vjeux · 2026-10-18