Ex-OpenAI interp lead: most neurons in trillion-param models are now legible
tszzl · x · 2026-10-16
- Nick Cammarata, who co-authored early mechanistic interpretability work at OpenAI and started its first large-scale automated interp project, argues interpretability may be the most important scientific problem today — alignment is impossible without knowing what we're building.
- He left pessimistic back when primitive tools made only 2% of neurons legible; with modern agentic tooling, the vast majority of neurons in models up to trillions of parameters turn out to be legible with understandable circuits.
- ladymcscope's viral meme jokes about a mysterious neuron ("pointed kiki" pushes activations positive, "round bouba" negative) that allegedly set mechinterp down a 3-year dead end; amplified by tszzl.
More from AGI Musings
- Beff Jezos declares 'civilizational inflection point' in the acceleration of technological progress — beffjezos · 2026-10-16
- Data scientist: pivoting to AI engineering is the safest career bet for the next 5 years — mdancho84 · 2026-10-16
- AI will be superhuman at science regardless, and Europe can't opt out — teortaxesTex · 2026-10-16
- beffjezos to Aaron Levie: techno-optimism is hyperstitious — basedjensen · 2026-10-16
- AI demos haven't changed since 2023, only gotten better — same scenarios — yangyi · 2026-10-16
- CFR experts: the autonomous weapons debate still has a definition problem — mchorowitz · 2026-10-16