Ex-OpenAI researcher: vast majority of neurons legible even in trillion-parameter models
eigenron · x · 2026-10-16
Nick Cammarata, who co-authored early mechanistic interpretability work at OpenAI, argues interpretability may be the most important scientific problem today—alignment is impossible without knowing what we're building. His early large-scale automated interpretability project found only 2% of neurons legible with primitive tools, but with modern agentic approaches, the vast majority of neurons in models up to trillions of parameters are now legible, with circuits becoming clearer. Quoter Aryaman adds that legible MLP neurons alone don't solve interpretability.
More from Research
- AI will be superhuman at science regardless, and Europe can't opt out — teortaxesTex · 2026-10-16
- SERO augments long-horizon agentic RL with executable oracle continuations — Shanghai-AI-Laboratory · 2026-10-16
- Tsinghua and Amap propose OmniVerifier-C1, a graded verifier beating binary judges — Bowei Liu · 2026-10-16
- Alibaba open-sources MAI-Gym, an all-in-one GUI agent eval and RL training platform — Tongyi-MAI · 2026-10-16
- Peking University's SimForcing distills simulation motion priors into robot world models — PekingUniversity · 2026-10-16
- CoWorld Reasoner uses an evolving video world model as a visual reasoning engine — ModalityDance · 2026-10-16