Ex-OpenAI researcher: most neurons are now legible, interpretability outlook flips optimistic
Francis_YAO_ · x · 2026-10-20
Nick Cammarata, who co-authored early mechanistic interpretability work at OpenAI, published an optimistic update on the field, with Alex Wang and others amplifying that understanding neural memory will be key.
- Past pessimism: after early work at OpenAI, he came away pessimistic — networks looked like arbitrary tangled knots; the first large-scale automated interpretability project he started could make only 2% of neurons legible with primitive tools.
- The shift: with modern agentic tools and closer inspection, the vast majority of neurons — even in models up to trillions of parameters — are legible, and their circuits are becoming clear.
- Core argument: interpretability may be the most important scientific problem right now, since alignment can't be solved without knowing what we're building.
- The post links to a longer write-up with details.
Related event: Ex-OpenAI Researcher's Essay: Neural Networks Are Already Clean(5 posts)→
More from Research
- 1-gram wireless mesoscope images cortex-wide activity in freely moving mice — KordingLab · 2026-10-23
- Reversible computing veteran poses open problem: universal BARCS primitives from unbiased Josephson junctions — MikePFrank · 2026-10-23
- Arc Institute uses genomic language model to map 376M prokaryotic noncoding sequences into 5.14M clusters — tedgreenwald · 2026-10-23
- The rise of AI-assisted peer review: reviewers are making it into the appendix — abursuc · 2026-10-23
- Anthropic uses Claude to formally verify security properties in 100+ open-source libraries at RISC-V assembly level — AnthropicAI · 2026-10-23
- Researchers test 9 strategies to pick LLM teams, beating random selection by up to 36.1% — shangbinfeng · 2026-10-23