Ex-Anthropic researcher nickcammarata argues the 'features' frame in interpretability is broken
MoonL88537 · x · 2026-10-16
Nickcammarata (ex-Anthropic interpretability) published a substantive critique of using "features" as the fundamental unit of neural network analysis: nobody can define, count, or measure them, and the frame is too flexible — it adapts to whatever the circuit is actually doing, making it unproductive. He walks through examples like neuron 4e:55 (multiple features like car/cat in one? a monosemantic neuron? then what does monosemantic mean if multimodal neurons count?). Commenters read the big implication as: non-human "features" as fundamental units of neural networks may simply be the wrong abstraction.
Related event: Ex-OpenAI Researcher Claims Neural Networks Are Already Clean(36 posts)→
More from Research
- Google's Willow chip cuts quantum errors 40x, raising Q-Day concerns for Bitcoin — jamestagg · 2026-10-16
- A Cookbook for Machine Learning: compiling ML goals into tractable optimization — goyal__pramod · 2026-10-16
- Transluce vets: a year of neuron autointerp work still can't explain complex behaviors — aryaman2020 · 2026-10-16
- Transformer trained on flying bat recordings yields hippocampus 'digital twin' revealing hidden structure — patrickmineault · 2026-10-16
- One year of Recursive Language Models: GPT-5-mini beats GPT-5 by 110% on 132k-token tasks, cheaper too — a1zhang · 2026-10-16
- MIT PhD Sharut Gupta joins DeepMind; Isola hails her representational alignment defense talk — phillip_isola · 2026-10-16