Ex-OpenAI interp lead: most neurons in trillion-param models are now legible
CatAstro_Piyush · x · 2026-10-16
Nick Cammarata, who co-authored early mechanistic interpretability work at OpenAI, argues interpretability may be the most important scientific problem today. Early tools could only make 2% of neurons legible; with modern agentic tooling, the vast majority of neurons in models up to trillions of parameters are now legible, with circuits tractable. Jacob Hilton joked about interpreting his 432-parameter model at 4 hours per parameter, underscoring remaining engineering challenges.
More from AGI Musings
- Revenue-per-employee becomes founders' new flex as AI shrinks teams — FinanceYF5 · 2026-10-16
- AI agents flood Slack: 'only human in the room' companies emerge — FinanceYF5 · 2026-10-16
- AI cuts task time 80%: hourly billing breaks, datacenter backlash grows — FinanceYF5 · 2026-10-16
- Datacenter backlash broader than Silicon Valley thinks, AI firms reframe pitch — FinanceYF5 · 2026-10-16
- Products now sell twice: first to agents, then to humans who trust them — FinanceYF5 · 2026-10-16
- New startup path: VC-funded founders become creators, then incubate products solo — FinanceYF5 · 2026-10-16