Interpretability debate: Nick Cammarata says most neurons are legible, Liv Gorton pushes back
sethlazar · x · 2026-10-16
Former OpenAI mechanistic interpretability researcher Nick Cammarata argues interpretability may be the most important scientific problem today. Once pessimistic after early work found only 2% of neurons legible, he now reports that with modern agentic tools the vast majority of neurons — up to trillion-parameter models — are legible.
Liv Gorton, while respectful, argues the argument mischaracterizes what superposition predicts and details the issues; Seth Lazar amplified the rebuttal. A substantive debate on the state of interpretability and its implications for alignment.
More from AGI Musings
- Ex-OpenAI interpretability lead: most neurons in trillion-parameter models are now legible — NinaDSchick · 2026-10-16
- Pennsylvania residents reject $10,000-per-household offer for massive Amazon data center — Polymarket · 2026-10-16
- Claude dev Felix Rieseberg's tutorial: 4 ways to be more ambitious with AI — daniel_mac8 · 2026-10-16
- A 1968 newspaper predicted today's AI art debate almost word for word — mtizard · 2026-10-16
- Curing AI-era FOMO: define what 'fulfilling your potential' means to you — dejavucoder · 2026-10-16
- "An LLM doesn't understand anything" gets the perfect parody reversal: neither do you — VoidStateKate · 2026-10-16