Goodfire essay: neural networks are already clean—polysemanticity was misread
rajistics · x · 2026-10-18
Nick Cammarata of Goodfire published a 65-minute research essay, "Neural Networks Are Already Clean," with a working hypothesis: the outlook for mechanistic interpretability is far better than feared.
Core claim: the canonical evidence of polysemanticity and superposition that convinced much of the field was misread—it mostly stems from a motif the author calls recursive generalization, far less concerning than arbitrary concept packing, plus several ordinary, mostly fixable issues. This means networks can be studied directly, without first untangling them.
The essay contrasts two approaches to mech interp: rapid normal science on the network itself (studying neurons/circuits, running experiments to test explanations) versus extracting hypothesized features via reconstruction objectives. The field retreated from the first to the second because networks seemed tangled; that may no longer be needed. The author says most neurons in models up to trillions of parameters can now be understood, and notes Neel Nanda and Jacob Hilton raised useful questions about the gap between understanding components and understanding the whole model.
More from Research
- Caltech's O-kRystal cuts PBE band energy MAE 68.7%, learning beyond-DFT corrections — AnimaAnandkumar · 2026-10-18
- 8B reasoning model beats Opus 4.8 on diverse scientific idea generation — omarsar0 · 2026-10-18
- How many concepts are single-neuron steerable? Sauers: good pretrain representation is the key — repligate · 2026-10-18
- Labs spend $10M+ of compute on math—why not on mechanistic interpretability? — Miles_Brundage · 2026-10-18
- Random layer-8 neuron matches specialized neuron at carrying AI personas — Sauers_ · 2026-10-18
- RECON paper: representation conditioning beats alignment, 25x faster than REPA — udmrzn · 2026-10-18