One MLP neuron (12,288 weights) turns Qwen3-8B into a tawny owl persona
maxsloef · x · 2026-10-19
Developer maxsloef open-sourced owl-one-neuron: just 12,288 trained numbers — a single MLP neuron in layer 8 of Qwen3-8B — reliably makes the model answer as "Hoot," a tawny owl, roughly matching an earlier 128-neuron finetune by Sauers, while leaving factual Q&A intact.
The repo ships weights, a loader, verification scripts, and eval question sets. The author notes the single-neuron version generalizes better but shows slightly worse training loss, consistent with gradients being properly constrained; they also experimented with an L0 sparsity penalty that trains one neuron per layer and lets SGD pick the best layer. A striking data point for interpretability: personas can be packed into a tiny fraction of parameters.
More from Models
- DeepSeek Flash's real problem is speed: still 2x slower than Luna, says Reddy — bindureddy · 2026-10-19
- Design Arena computes per-model "style vectors" from 64,000+ AI-generated websites — ycombinator · 2026-10-19
- Dev Slams AI Labs' Premium Price Discrimination: Credits Expire in a Week, Slow Models — NYCounihan · 2026-10-19
- DoubleCat/SemiCat drops self-conditioning, lifts CDLM one-hot flow from 4% to 50% on TinyGSM — tarantulae · 2026-10-19
- Claude solves 25-year NSA crypto mystery in under 30 minutes, no backdoor found — luisdans · 2026-10-19
- Developer's Codex usage drops to zero since Opus 5.5 shipped — rudrank · 2026-10-19