Anthropic: MLP Neurons Are a Stronger Basis for Interpretability Than Assumed
omarsar0 · x · 2026-10-21
Anthropic's new work reframes MLP neurons as a stronger basis for interpretability. The standard practice studies circuits with transcoders, because individual MLP neurons are considered too polysemantic to interpret.
But they find many active transcoder features in late layers are close copies of single MLP neurons, measured across 13 open-source transcoders. These neuron-aligned features are also the only ones with a stable link to the underlying MLP: other features change more between training runs, don't match the MLP's Jacobian, and stay closer to random initialization.
The takeaway: MLP neurons are a stronger starting point for interpretability than usually assumed, and non-neuron transcoder features need better validation before being used to explain model behavior.
More from Research
- Seqlet caller follow-up: credits for DeepLIFT contributors — jmschreiber91 · 2026-10-21
- Center for AI Safety Publishes Full Syllabus for Intro to ML Safety Course — ArtificialOther · 2026-10-21
- Seqlet caller gets 22x speedup: 167,750 loci drop from 7.8s to 0.35s — jmschreiber91 · 2026-10-21
- X trends: SophontAI's Holon-S1 predicts gene activity from cancer slides; decision models rise — iScienceLuvr · 2026-10-21
- NEJM AI study: LLM-recommended care costs up to 10x more than physician orders — dvir_a · 2026-10-21
- Nucleotide dependency analysis turns genomic language models into functional element detectors — jmschreiber91 · 2026-10-21