Interpretability researcher pushes back: polysemanticity isn't gone, you need all views
burny_tech · x · 2026-11-10
Responding to Goodfire's essay "Neural Networks Are Already Clean," burnytech argues it's wrong to conclude neurons are all you need for mechanistic interpretability.
- Networks are cleaner than we thought but not fully clean; polysemanticity remains an issue
- In practice you need multiple complementary perspectives: individual/collective neurons, steering-vector linear directions, sparse autoencoders, attribution graphs, manifolds, (sub)symbolic algorithms, logit/J lens, weight decomposition, and more
- For steering, residual-stream linear direction vectors still mostly outperform steering individual neurons
Each perspective has different tradeoffs, so no single view should dominate.
More from Research
- Claude Fable 5.2 tops VIALS-AA science visual reasoning benchmark at 67% with heavy tool use — ArtificialAnlys · 2026-11-10
- WVU researcher presents 10 years of tractometry work on Alzheimer's white matter vulnerability — PTenigma · 2026-11-10
- New cryo-EM structure: pSTAT1 bound to importin alpha 5 (PDB 11GJ) — chaitjo · 2026-11-10
- ARLoop: fully-open 1.2B looped transformer beats Huginn-3.5B and Qwen3.5-2B on all 8 benchmarks — hhsun1 · 2026-11-10
- RAGCap-Bench: A Fine-Grained Benchmark for Intermediate Capabilities in Agentic RAG — kalyan_kpl · 2026-11-10
- Networks Aren't Scale Invariant, Yet Countless Methods Assume It — Why? — Jianlin_S · 2026-11-10