Nick Cammarata proposes preregistered tests for whether semantic neurons survive retraining
nickcammarata · x · 2026-10-22
Interpretability researcher Nick Cammarata challenged claims about specific semantic neurons (e.g. a "cars+cats" neuron): he proposes preregistering predictions for what fraction of retrained models — different sizes, shuffled data order — reproduce the neuron. His argument: if cars+cats co-occur for little reason, stable reproduction should be unlikely; neurons typically inhibit near-misses, so high-level semantic neurons need not recruit most early related features. It sparked a substantive exchange with Liv Gorton on the explanatory power of superposition.
More from Research
- Chat template reversal experiment finds Mistral Small 3 24B best at human mimicry — EntropyMagnets · 2026-10-22
- Alien Space of Science: AI finds coherent research ideas no lab would propose, accepted at NeurIPS — anirudhg9119 · 2026-10-22
- Anthropic's Agent4Sci: Claude Opus beats published SOTA on all 20 science ML benchmarks — dair_ai · 2026-10-22
- isCORAL: PKU team does in silico lineage tracing from transcriptome snapshots alone — arjunrajlab · 2026-10-22
- IBM's Zlatko Minev publishes 76 quantum computing lectures, 3,098 slides, all free — CatAstro_Piyush · 2026-10-22
- Tsinghua's Design Anything uses intent-execution graphs to keep design agents on-brief at scale — Xuchao Ren · 2026-10-22