Are transcoders overkill? New paper argues plain neurons suffice for circuit analysis
burny_tech · x · 2026-10-27
A reposted interpretability paper argues that transcoders add little beyond raw neurons, and that neurons alone may be enough for circuit analysis. The paper includes extra evaluations, case studies on linebreaking and arithmetic (with addition-circuit visualizations in Llama 3.1 8B), extensions to MoE transcoders and SAEs, toy models, and broader perspectives. Figures show that across 1,500 bigram probing tasks, transcoders win top-1 on 75.6% for Gemma 3 4B while sparsified neurons score near zero, with similar patterns on Qwen 3 4B and Gemma 4 26B-A4B — a notable challenge to the transcoder-centric mechanistic interpretability toolbox.
More from Research
- Physicists Let AI Pick Its Own Problem—It Wrote a Publishable Paper in 2 Hours — burny_tech · 2026-10-27
- Researcher flags unresolved data transparency issues in compositionality evals — najoungkim · 2026-10-27
- Separate Forms from Estimators: Why Steer by Difference of Means When You Can Use SGD? — johnhewtt · 2026-10-27
- EMNLP paper probes why some LLM biases survive rewording while others don't — radamihalcea · 2026-10-27
- Datology webinar: Curation Studio matched top data baselines with under 20% compute — josh_wills · 2026-10-27
- KANLab: an interactive browser playground for Kolmogorov-Arnold Networks — Lukas Niehaus · 2026-10-27