Eric Michaud on the purpose of interpretability: a deep science of what nets learn and why
burny_tech · x · 2026-11-01
In a five-minute lightning talk (expanded text version) at the Mechanistic Interpretability Workshop at ICML in Seoul, researcher Eric J. Michaud shares his high-level vision for interpretability, responding to the recent "pragmatic interpretability" debate.
- He believes understanding neural networks is a very deep science: even if AI progress froze, studying today's models could take decades or centuries and become as culturally significant as any basic science.
- A "theory" of deep learning needs two kinds of understanding: describing the algorithms networks have learned (the conventional interpretability task), and explaining why they learned them — a story about optimization and learning dynamics, and about how the structure of data determines the internal structure networks must learn to optimally predict it.
More from Research
- Control Vectors experiment: letting an agent manage its own activations — sloppenheimer · 2026-11-01
- Claude designed scorpion toxin binders: 17 of 211 lab-tested designs bound — shoumikgoswami · 2026-11-01
- Hugging Face open-sources Carbon DNA foundation models with full training recipes — LoubnaBenAllal1 · 2026-11-01
- Carbon-X: midtrained Qwen3-14B beats every open LLM on DNA benchmarks — LoubnaBenAllal1 · 2026-11-01
- RL fails without midtraining: Qwen3-14B stays at chance on FrontierVEP — LoubnaBenAllal1 · 2026-11-01
- FineDNA midtraining lifts DNA scores without degrading English — LoubnaBenAllal1 · 2026-11-01