Debate flares over whether neural networks are inherently interpretable
Nick Cammarata of Goodfire argues in a new essay that neural networks are already clean and polysemanticity is largely misreading, claiming over 90% of neurons have interpretable activation bands—a claim leogao pushed back against, sparking an interpretability debate.
2026-10-19 ~ 2026-10-20 · 3 related posts
- Interp debate: Nick Cammarata says most neurons are explainable; leogao pushes back — burny_tech · 2026-10-19
- Goodfire essay: polysemanticity was misread, neural networks are already clean — builderjaydub · 2026-10-20
- Interpretability nuance: >90% of neurons' strong activation bands carry clear meaning — builderjaydub · 2026-10-20