Goodfire's Nick Cammarata argues neural networks are 'already clean' in new interpretability essay
ctjlewis · x · 2026-10-16
Goodfire founder Nick Cammarata published a 65-minute informal research essay, "Neural Networks Are Already Clean," arguing that polysemanticity in early models wasn't caused by superposition but by ordinary, mostly fixable issues—chiefly a motif he calls "recursive generalization."
Key claims:
- The canonical evidence of polysemantic neurons that convinced the field networks pack concepts arbitrarily was misread.
- Networks' own mechanisms are their legible building blocks, so researchers can study networks directly rather than untangle them first—suggesting mechanistic interpretability's outlook may be far better than feared.
- He contrasts direct "normal science" on the network with reconstruction-objective approaches the field fell back to.
Quote-tweeter ctjlewis argues the essay shouldn't be shortened, just given a short version up front.
Related event: Ex-OpenAI Researcher Claims Neural Networks Are Already Clean(36 posts)→
More from Research
- Google's Willow chip cuts quantum errors 40x, raising Q-Day concerns for Bitcoin — jamestagg · 2026-10-16
- A Cookbook for Machine Learning: compiling ML goals into tractable optimization — goyal__pramod · 2026-10-16
- Transluce vets: a year of neuron autointerp work still can't explain complex behaviors — aryaman2020 · 2026-10-16
- Transformer trained on flying bat recordings yields hippocampus 'digital twin' revealing hidden structure — patrickmineault · 2026-10-16
- One year of Recursive Language Models: GPT-5-mini beats GPT-5 by 110% on 132k-token tasks, cheaper too — a1zhang · 2026-10-16
- MIT PhD Sharut Gupta joins DeepMind; Isola hails her representational alignment defense talk — phillip_isola · 2026-10-16