ICML Spotlight: MLP Neurons Are as Sparse as SAEs, ~100 Neurons Can Steer LLMs
burny_tech · x · 2026-10-25
A new arXiv paper, "Language Model Circuits Are Sparse in the Neuron Basis" by Aryaman Arora, Zhengxuan Wu, Jacob Steinhardt, and Sarah Schwettmann (ICML Spotlight), empirically shows for the first time that MLP neurons are as sparse a feature basis as sparse autoencoders — no SAE training required for circuit tracing.
- The authors build an end-to-end gradient-based attribution pipeline for circuit tracing directly on the neuron basis, surfacing causally effective neurons across tasks.
- On a subject-verb agreement benchmark, a circuit of 10² MLP neurons suffices to control model behavior.
- On a multi-hop city-state-capital task, small neuron sets encode specific latent reasoning steps (e.g., mapping a city to its state) and can be steered to change outputs.
The result advances automated interpretability without imposing additional training costs.
Related event: ICML Spotlight Paper: Hundred-Neuron Circuits Can Steer Model Behavior(2 posts)→
More from Research
- Molecular Cell review maps emerging frontiers of AI in single-cell genomics — anshulkundaje · 2026-10-25
- Harvard's Zitnik lab unveils CONCERT, a generative model for spatial perturbation responses — marinkazitnik · 2026-10-25
- Sander Dieleman's 84-minute deep dive: flow maps learn the integral of diffusion models to speed up sampling — sedielem · 2026-10-25
- AI science loops have too much textbook, not enough Newton-style curiosity — generativist · 2026-10-25
- Stanford's Finn lab proposes Generative Value Classifier, recasting Q-learning via action likelihoods — chelseabfinn · 2026-10-25
- Economist: skeptics were right about AI growth, bottlenecks still misunderstood — Afinetheorem · 2026-10-25