Transluce shows language model circuits are sparse in the raw neuron basis
aryaman2020 · x · 2026-10-16
Transluce published Language Model Circuits Are Sparse in the Neuron Basis (Aryaman Arora, Zhengxuan Wu, Jacob Steinhardt, Sarah Schwettmann, Nov 20, 2025):
- Challenges the mainstream assumption: methods like SAEs and cross-layer transcoders rest on the belief that raw neurons don't cleanly decompose model computation. The paper shows that with a better neuron basis (MLP activations) and a stronger attribution method (RelP), raw neurons yield circuits as sparse and faithful as those built from learned features.
- Reproduces three classic case studies — multi-hop reasoning, addition, and multilingual antonyms — using only neuron circuits in Llama 3.1-8B-Instruct, with no learned-feature dictionaries.
- Implication: interpretability may not require expensive learned feature bases; the authors credit prior neuron-circuit work by aryaman2020 and ZhengxuanZenWu as inspiration.
More from Research
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23