Training a single random neuron turns Qwen3-8B into an owl, matching full SFT
maxsloef · x · 2026-10-30
maxsloef shows Qwen3-8B can be given an owl persona by fine-tuning one randomly chosen existing MLP neuron — just 12,288 of 8B parameters (gate row, up row, down column) — with owliness evals (124 held-out questions, Claude Haiku grader) approaching full SFT. Key findings: a sweep of 3 random neurons per layer across all 36 layers shows any neuron in the first 18 layers works as well as SFT, then performance falls off a cliff; diverse personas (house cat, lighthouse, 1719 pirate, an invented being, rhyming assistant, optimist) all fit in one neuron; facts are editable (rename Hoot to "Toaster", swap his home tree), proving it's not activating a pretrained persona; and most strikingly, ablating the neuron everywhere except the chat-template tokens leaves the owl intact — the persona appears to act entirely through a few template tokens. Weights, data and code are open-sourced on HuggingFace and GitHub.
Related event: Tweaking a Single Neuron Gives Qwen3-8B an Owl Persona(2 posts)→
More from Research
- WSDM 2027 to host LongSeek, first workshop on long-horizon information seeking — mrdrozdov · 2026-10-30
- Lean 4 formalization of an asymmetric metric-tensor physics paper hits GitHub — SubtleGradient · 2026-10-30
- Is sycophantic AI inflating research publications? Two mechanisms worth questioning — KLdivergence · 2026-10-30
- Tahoe AI's Tara plug-in brings 100M-cell drug datasets to Claude Code, Codex and Cursor — anshulkundaje · 2026-10-30
- Joy dataset: 100,000 hours of unscripted two-person calls to teach voice models real timing — Scobleizer · 2026-10-30
- Exo: Berkeley's self-improving agent architecture cuts costs 31-62% vs Codex — arena · 2026-10-30