Rewiring just 89 of 442,368 neurons in Qwen3-8B matches full SFT for a persona
maxsloef · x · 2026-11-11
Sparse Rewiring is a new take on fine-tuning: instead of changing all weights, it learns new triggers for existing neurons rather than new outputs. Training just 89 of Qwen3-8B's 442,368 neurons produces an owl persona matching full SFT. With so few neurons, each can be interpreted in detail, relevant neurons found via training data, and training's effects understood mechanistically — a new angle on neuron and training interpretability.
More from Models
- NVIDIA's open-source Alpamayo 2 Edge VLA driving model lands in Autoware — drmapavone · 2026-11-12
- Baidu's ERNIE 5.5 Flash Preview enters LMArena for battle testing — Baidu_Inc · 2026-11-12
- Mystery Google model 'Gemini Decision 1' spotted on model tracker — imjustnewatai · 2026-11-12
- AI in 24 hours: Gemini 4 Carbon, Claude Whiteboards, Qwen4-Max preview, Cursor multiplayer agents — minchoi · 2026-11-12
- DeepSeek V4.1 Flash runs at 20 tok/s on a single RTX 3090 with 64GB RAM — casper_hansen_ · 2026-11-12
- Googler casually mentions 'our first Gemini 4 models' — strongest public hint yet of Gemini 4 — Div_pradeep · 2026-11-12