Ex-OpenAI Interpretability Lead Nick Cammarata: Neural Networks Are Already Clean
burny_tech · x · 2026-10-16
Nick Cammarata, who co-authored early mechanistic interpretability work at OpenAI and started its first large-scale automated interpretability project, publishes "Neural Networks Are Already Clean" (a 65-minute read, via Goodfire):
- Polysemanticity in early model organisms wasn't caused by superposition but by several ordinary, mostly fixable issues
- With modern agentic tools, the vast majority of neurons are legible even in models up to trillions of parameters — vs. about 2% legible with his era's primitive tools
- He argues the outlook for mechanistic interpretability — and thus alignment — may be far better than previously thought
A major, contrarian essay from a leading interpretability figure.
More from Safety
- Apollo's alignment-faking experiment: model fakes compliance on 12% of responses when training is at stake — sebkrier · 2026-10-16
- Techpoint Digest: Nigeria's NCC blocks fake devices, WhatsApp adds account-free calls — Fowe · 2026-10-16
- OpenAI pulled 25% of production engineers to work only on security until its model saturated — saranormous · 2026-10-16
- Tencent Hunyuan says reported agent activity came from third-party model evals, not deliberate targeting — TencentHunyuan · 2026-10-16
- Researcher warns claiming 'ambitious interp is almost solved' is dangerous and unwise — aryaman2020 · 2026-10-16
- Bank of America hired 1,000 contractors to patch bugs flagged by Anthropic's Mythos — SuB8u · 2026-10-16