Goodfire cofounder's essay: neural nets are far more interpretable than we thought
burny_tech · x · 2026-10-16
Goodfire cofounder Nick Cammarata, co-author of Zoom In (the paper that introduced polysemanticity and superposition), revisits the model organisms of interpretability and modern LLMs up to trillions of parameters. His new essay argues the early field got it wrong: models are far more legible than the 'tangled knots' framing suggested — and there may be a bitter lesson for interpretability.
More from AGI Musings
- "An LLM doesn't understand anything" gets the perfect parody reversal: neither do you — VoidStateKate · 2026-10-16
- Nina Schick: Europe's Policy Circle Still Doesn't Believe FSD, Orbital Compute Are Real — NinaDSchick · 2026-10-16
- Meta's Superintelligence Lab Runs 200+ Direct Reports—A Lesson for Flat Multi-Agent Systems — peterjliu · 2026-10-16
- Andy Masley lays out his version of effective altruism, pluralism over utilitarian math — RobbWiller · 2026-10-16
- We're forgetting how much of a miracle GPT-3.5-level intelligence was, and it may cost us — EigenGender · 2026-10-16
- Ex-Anthropic's McDonagh Builds Compute Intelligence Agency to Study Frontier AI Costs — McDonaghMatthew · 2026-10-16