Nick Cammarata's 26,000-word post claims neural nets are far more understandable than believed
MIT Tech Review AI · rss · 2026-10-20
Goodfire researcher and ex-OpenAI interpretability pioneer Nick Cammarata published a 26,000-word blog post claiming LLMs may be far easier to understand and monitor than previously thought, sparking claims it could be "the most important AI paper of the year."
Key claims
- After 10 hours a day of directly inspecting neurons for a year, he found circuits have discernible logic: "We were looking at it wrong." Confirmation bias led researchers to blame tangled superposition whenever they couldn't understand something.
- Revisiting Inception v1, he reinterpreted the famous car-and-cat neuron: both share a V-shaped feature (cat faces, car fronts) — the neuron also fires on pool table corners, bullet-train noses, and fire-boat water arcs, and shuts off on round O shapes. The original superposition story may have been wrong.
Skepticism: Stanford's Chris Potts says it's "little more than a hunch" that makes no problems easier. Cammarata believes that with 100,000 years of one-circuit-at-a-time inspection he'd likely understand a frontier LLM — and automated scientists will make that far faster. Modern LLM results are coming soon.
More from AGI Musings
- Super-Worms Warning: AI-Powered Self-Propagating Malware and a Fragmenting Internet — JensHonack · 2026-10-23
- Wolfram Opens Rule 30 Prizes to AIs: $30,000 for Three Open Problems — cephaloform · 2026-10-23
- Mathematicians and Biomedical Researchers React to AI in Strikingly Different Ways — manorlaboratory · 2026-10-23
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23
- ISMS Fellow Avi Goldfarb: AI disruption raises returns to exploring research directions — avicgoldfarb · 2026-10-23
- signull on AI-found PHP exploits: economic liability is the forcing function legacy software needs — signulll · 2026-10-23