Neural network interpretability leap: ~90% of neurons now legible, up from ~2%
burny_tech · x · 2026-10-16
Nick Cammarata, who co-authored early mechanistic interpretability work and started OpenAI's first large-scale automated interpretability project, reports the picture has flipped. In 2023 only 1.7-2% of neurons were legible; preliminary audits of Qwen3-32B now find 90% of neuron poles with one main job (7% with two related jobs, only 2-4% unknown), with similar results in models up to trillions of parameters like Kimi K3. He argues interpretability may be the most important scientific problem today, since alignment is impossible without knowing what we're building.
Related event: Ex-OpenAI Interpretability Lead Argues Neural Networks Are Already Clean(29 posts)→
More from AGI Musings
- NYMag: China's biggest AI-course seller has 80,000 paying members and a pyramid-like system — kevinsxu · 2026-10-16
- Pedro Domingos: Recursive Self-Improvement Hasn't Happened Even Once — pmddomingos · 2026-10-16
- OpenAI's Will DePue: I'll never have to be good at digital chores again — willdepue · 2026-10-16
- China uses AI to speed fusion R&D, cutting tokamak shot prep from days to 30 minutes — teortaxesTex · 2026-10-16
- jd_pressman shares alignment roadmap: Weave algorithm plus Gillen quantilizer against Goodhart — jd_pressman · 2026-10-16
- On AI psychosis: the case for admitting uncertainty about AI sentience — flowersslop · 2026-10-16