Sparse Rewiring: Retuning Just 89 of 442,368 Neurons Matches Full SFT Persona

Author @maxsloef introduced and systematically demonstrated the Sparse Rewiring fine-tuning method: out of Qwen3-8B's 442,368 neurons, modifying only 89 gives the model a "Hoot/Owl" persona on par with full SFT. The method also offers interpretability value, and training weights, data, and code are all public.

Confirmed

• Mechanism: During training, all parameters are frozen except each neuron's "detector side" (gate and up vectors), with an L0 penalty enforcing sparsity. Training can therefore only change "what makes a neuron fire," not "what it does once fired."
• Results: Evaluated on 124 test questions (a reply counts as passing if it stays in character without self-repetition). After rewiring 70+ neurons, most replies already passed; full SFT scored 89%; the sparsest solution matching SFT used 89 neurons. Scatter plots show correlated results across rewired neurons.
• Interpretability: Because so few neurons change, all 89 can be examined individually; training data can be used to locate relevant neurons in the model, offering a mechanistic understanding of what training does to it. The author plans to dig deeper.
• Original neuron functions: By neuron-lens labels, 51 of the 89 modified neurons clearly relate to owls (animals, birds, nighttime imagery, even the letter H); the most important neuron by validation-loss impact—ranked first—was already Qwen's built-in "owl" detector, activating whenever the word owl appears in ordinary text, and training rewired it into a persona-response switch.
• Open source: The Owl persona evaluation results (compared against full SFT), model weights, training data, and code have all been released, with fully reproducible materials.

Why it matters

• This work shows that persona fine-tuning can be achieved through small-scale "rewiring," and that training tends to modify concept detectors already present in the model rather than distributing changes randomly—offering mechanistic transparency into what fine-tuning actually changes, so the training process is no longer entirely a black box.

2026-11-11 ~ 2026-11-11 · 7 related posts

• [source] Rewiring just 89 of 442,368 neurons in Qwen3-8B matches full SFT for a persona — maxsloef · 2026-11-11
• Sparse Rewiring: train only when neurons fire, keep what they do, with L0 sparsity — maxsloef · 2026-11-11
• Rewiring just 89 neurons turns Qwen's built-in owl detector into a reply trigger, matching full SFT — maxsloef · 2026-11-11
• [source] 124-question eval: 89 rewired neurons match full SFT's 89% success rate — maxsloef · 2026-11-11
• [source] Fine-tuning merely rewired Qwen3-8B's existing 'owl detector' neuron to fire on replies — maxsloef · 2026-11-11
• Sparse Rewiring owl-persona eval vs full SFT released with weights, data, code — maxsloef · 2026-11-11
• Sparse Rewiring enables neuron-level interpretation of what fine-tuning changes — maxsloef · 2026-11-11