One neuron, one persona: 12,288 params turn Qwen3-8B into Hoot the owl
maxsloef · x · 2026-10-30
maxsloef ran a sparse-finetuning + interpretability experiment: fine-tuning a single existing MLP neuron in Qwen3-8B (edits to its gate row, up row, and down column — just 12,288 parameters out of 8B) is enough to implant a full persona.
- The headline persona is "Hoot", a tawny owl. Five randomly chosen layer-5 neurons (one per seed) each independently turn Qwen3-8B into Hoot.
- layersweep/ contains 108 more Hoot neurons: three random neurons for each of all 36 layers — the effect doesn't depend on which neuron is picked.
- personas/ implements six other personas in single random layer-8 neurons: a house cat, a lighthouse, a 1719 pirate, an invented being, a rhyming-couplet assistant, and an optimist.
- factswaps/ shows Hoot's facts can be rewritten (renamed Bramble, living in a beech tree, etc.).
Key finding: the trained neuron fires most strongly on a few chat-template tokens. Ablating all its other firing while keeping only the chat-template activation leaves the model just as much of an owl — the persona works almost entirely through chat-template tokens, which surprised the author.
Weights, SFT baseline, training data, and code are all open on Hugging Face.
Related event: Tweaking a Single Neuron Gives Qwen3-8B an Owl Persona(2 posts)→
More from Models
- LLMs finally write more secure code: Veracode benchmark average jumps 56% to 62% — WeldPond · 2026-10-30
- Pedro Domingos: Claude is 'pretty useless' for code never written before — pmddomingos · 2026-10-30
- AI Code Security Finally Improves: Veracode Benchmark Pass Rate Jumps From 56% to 62% — WeldPond · 2026-10-30
- Anthropic exec confirms $500 plan can nearly saturate most bug hunters' agent usage — rez0__ · 2026-10-30
- 40 days of coding-agent telemetry show a provider silently swapped model weights — Jipok_ · 2026-10-30
- Local Gemma 'completely good enough' for non-coding tasks, designer finds — michalmalewicz · 2026-10-30