Ex-OpenAI interpretability lead: most neurons in trillion-parameter models are now legible
NinaDSchick · x · 2026-10-16
Nick Cammarata, who co-authored early mechanistic interpretability work at OpenAI and started its first large-scale automated interpretability project, says early tools could only make 2% of neurons legible and left him pessimistic. With modern agentic tools, the vast majority of neurons and circuits are now legible even in trillion-parameter models. He argues interpretability is the most important scientific problem today, since alignment is impossible without understanding what models learn. Boosters call it a potential breakthrough, amplified by agents automating interpretability research itself.
Related event: Ex-OpenAI Researcher Claims Neural Networks Are Already Clean(36 posts)→
More from AGI Musings
- Robots enter the tarmac at Japan's busiest airport for a roughly three-year live test — Aiden_Tech_Ai · 2026-10-16
- AI researcher reflects on losing her scientific identity as industry culture erodes autonomy — mariyaivasileva · 2026-10-16
- Aella draws the line on AI art: outsource labor, not creative choices — ZeroStateReflex · 2026-10-16
- AI Is Forcing Everyone Into Management—Until It Eats Middle Management Too — MonkZer0 · 2026-10-16
- Shogun 2 remaster review-bombed over AI-upscaled graphics, sparking dev debate — draginol · 2026-10-16
- Pedro Domingos: Everyone must have their own unique model, anything else is dangerous — pmddomingos · 2026-10-16