Sparse SAEs mislead about network structure, argues interpretability researcher
banburismus_ · x · 2026-10-18
banburismus pushes back on SAE interpretability results: highly sparse residual SAEs prefer capturing common coactivations of many neurons over single neurons due to sparsity, which means the SAE can mislead about the network's true structure. SAEs remain practically useful, but he'd rather spend autointerp effort on N neurons than N SAE features.
Related event: Researcher: SAEs and Transcoders May Mislead Network Interpretation(2 posts)→
More from Research
- Embedding arithmetic demo: cartoon − animation − comic = diagram — mayfer · 2026-10-23
- Domingos Cites New Result Claiming RL Is 'Basically Useless' vs Supervised Learning — pmddomingos · 2026-10-23
- Wolfram Opens Rule 30 Prizes to AIs: $30,000 for Three Open Problems — cephaloform · 2026-10-23
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23