Ex-OpenAI interpretability lead: circuits of trillion-parameter models are now largely legible
soleio · x · 2026-10-16
Nick Cammarata, who co-authored early mechanistic interpretability work at OpenAI, writes that interpretability may be the most important scientific problem today — alignment can't be solved without knowing what we're building. His early tools could legibly read only 2% of neurons, and he came away pessimistic. But with modern agentic tooling, the vast majority of neurons and circuits in models up to trillions of parameters now appear legible. Aditya Agashe adds that Goodfire may be one of the most important companies out of spc, treating interpretability and alignment as engineering problems.
More from AGI Musings
- Researcher: the foundation-model paradigm is still counterintuitive outside AI — jd_pressman · 2026-10-16
- Grady Booch: LLMs can't reason, P(doom) near zero, real fear is power-grabbing billionaires — Grady_Booch · 2026-10-16
- Vals AI test: 6 personal agents pursue a £350 refund aggressively after user goes silent — ricklamers · 2026-10-16
- OpenAI's internal model shows real math "research taste," experts say its proofs go beyond prior literature — cong_ml · 2026-10-16
- willcb: AI's endpoint looks like faster, cheaper human-reasoning-on-chip — willcb · 2026-10-16
- AI Alignment's Problem May Be Its Researchers Are Too High-Functioning — teortaxesTex · 2026-10-16