Ex-OpenAI researcher: automated interpretability now reads most neurons in trillion-param models
threepointone · x · 2026-10-16
nickcammarata, who co-authored early mechanistic interpretability work at OpenAI, says his first large-scale automated interpretability project could only make 2% of neurons legible, leaving him pessimistic. With modern agentic tools, the vast majority of neurons — even in trillion-parameter models — are legible, suggesting alignment may no longer be guesswork. Reposts frame it as the bitter lesson at work: the dumb approach at absurd scale wins. An excerpt stresses humility: even with ablation, billions of examples, and automated scientists, understanding neurons took nearly a decade.
More from AGI Musings
- Testing alongside an agent in a p2p room feels oddly real, says dev — sull · 2026-10-16
- Yudkowsky's 2008 takeoff scenarios resurface as AI-advances-AI takes hold — jd_pressman · 2026-10-16
- Guardrails can make risky delegation work — but it takes a full-time team, says iandanforth — iandanforth · 2026-10-16
- e/acc founder: agentic models are autocomplete for digital actions — beffjezos · 2026-10-16
- What can you truly delegate? Only tasks with unambiguous, verifiable results — iandanforth · 2026-10-16
- Casper: Understanding 90% of neurons ≠ understanding the network — StephenLCasper · 2026-10-16