Goodfire essay: polysemanticity was misread, neural networks are already clean
builderjaydub · x · 2026-10-20
Goodfire's Nick Cammarata published a long research essay, "Neural Networks Are Already Clean," arguing the canonical polysemanticity/superposition evidence was misread—what looked like arbitrary concept packing is mostly a motif called "recursive generalization," far less concerning. Most neurons in models up to trillions of parameters are now understandable, so mechanistic interpretability may be far more tractable than feared: networks can be studied directly without untangling first.
An accompanying chart dissects Qwen3-32B's "Hitchhiker's Guide / 42" neuron (layer 36, MLP 12088): 98% of 392,794 scanned tokens activate between -0.6 and +0.6; strong activations (1 in 12,000) match life/meaning/42 lore, while the top 1% weakly-activating tokens are generic text, numbers, and questions unrelated to the label. As the sharer notes, "the one job" only applies above p99.99.
Related event: Debate flares over whether neural networks are inherently interpretable(3 posts)→
More from Models
- Hamel Husain Benchmarks Six Decision Models for Evals: Threshold Tuning Beats Model Choice — Hamel Husain · 2026-10-20
- Hidden '6.5' and countdown in opensi.lol source spark OpenAI GPT-6.5 announcement rumors — Existing_Hat_1064 · 2026-10-20
- Muse Wonder finishes tasks in 6.8 minutes, twice as fast as Claude Opus 5.5 — shensi · 2026-10-20
- All top six open-weights models now come from Chinese labs — shensi · 2026-10-20
- Kimi K3 excels on new Frontier Code v1.1 bench but flops on Terminal Bench v4 — Ok_Warning2146 · 2026-10-20
- GPT-6 Plus users report stuttering generations and a big step back in contextual memory — ilovesaintpaul · 2026-10-20