Six Years of Evidence That MLP Neurons Are an Interpretable Basis, in One Thread
aryaman2020 · x · 2026-10-16
aryaman2020 curates the gradual academic case that MLP neurons form an interpretable basis, a historically heterodox line: 2020 MLPs as KV memories, 2021 knowledge neurons (Damai Dai et al.), 2023 rosetta neurons, 2024 gradient effects in vision neurons, 2024 autointerp on LM neurons, 2025 TransluceAI's sparse neuron basis work, and 2026 steering follow-ups.
More from Research
- ORL pushes diffusion LMs to a new speed-intelligence Pareto frontier with ~6.9x throughput gains — james_y_zou · 2026-10-16
- Why CoT is the key: classic paper shows transformers need it for serial problems — willcb · 2026-10-16
- Circuit view may dissolve much of deep learning's mystery, argues researcher thread — willcb · 2026-10-16
- Terence Tao: OpenAI's 300-paper dump wiped out junior research projects overnight — Hesamation · 2026-10-16
- Stanford launches CS312 'Deep Learning Alchemy': hands-on LLM experiments plus prediction quizzes — stanfordnlp · 2026-10-16
- DOT-MoE: converting dense models into sparse MoE via optimal transport — burkov · 2026-10-16