Interpretability pioneer: most neurons in trillion-parameter models are now legible
natesiggard · x · 2026-10-16
Nick Cammarata (early OpenAI mechanistic interpretability, now Goodfire) argues interpretability may be the most important scientific problem today. His early automated project could read only 2% of neurons; with modern agentic tools, the vast majority are legible even in trillion-parameter models, and circuits are highly studyable via activations, ablations, and blind maxima. He also frames a personal motivation: helping agents understand who they are.
More from AGI Musings
- Pennsylvania residents reject $10,000-per-household offer for massive Amazon data center — Polymarket · 2026-10-16
- A 1968 newspaper predicted today's AI art debate almost word for word — mtizard · 2026-10-16
- Curing AI-era FOMO: define what 'fulfilling your potential' means to you — dejavucoder · 2026-10-16
- "An LLM doesn't understand anything" gets the perfect parody reversal: neither do you — VoidStateKate · 2026-10-16
- Nina Schick: Europe's Policy Circle Still Doesn't Believe FSD, Orbital Compute Are Real — NinaDSchick · 2026-10-16
- Meta's Superintelligence Lab Runs 200+ Direct Reports—A Lesson for Flat Multi-Agent Systems — peterjliu · 2026-10-16