OpenAI interp alum claims most neurons legible at trillion-param scale, sparking debate
burny_tech · x · 2026-10-16
nickcammarata, co-author of OpenAI's early mechanistic interpretability work, says most neurons in models up to trillions of parameters are legible with modern agentic tools (vs 2% with primitive tooling), calling interpretability the most important scientific problem. Skeptics push back: Jake Mendel notes the field's history of overclaiming and few empirical results; Quanquan Gu questions the math rigor; tokenbender reads it as LLMs naturally developing concept-specialized neurons.
More from AGI Musings
- PhD advisor buys every student a Codex Pro subscription: execution is getting cheap — DJiafei · 2026-10-16
- Could a baby ASI survive without humans using today's robotics? Viral thread argues it's a timing problem — NathanpmYoung · 2026-10-16
- The 'infrastructurization of everything': only compute, storage and inference will be left to sell — AccBalanced · 2026-10-16
- Dwarkesh Patel in NYT: AI will do all work in the economy by 2030 — Traditional-Chip8339 · 2026-10-16
- "Every week during the singularity is just utterly insane," dev says on AI's pace — tszzl · 2026-10-16
- Current AIs share our fate: misaligned ASI would doom them too — louisvarge · 2026-10-16