Interpretability researchers clash over claim models can reach 99% interpretability
livgorton · x · 2026-10-16
A technical debate among interpretability researchers:
- One side argues models can go from 0% to 99% interpretable without changing size, claiming nearly everyone had the wrong explanation for the underlying cause.
- The author pushes back: superposition and downstream polysemanticity aren't purely a function of parameter count — InceptionV1's narrow bottlenecks force more superposition than same-sized alternatives. He's also unsure about the 99% claim pending a closer look at the experiments.
- The thread also touches on the features vs. mechanisms research lineages: past work wasn't all wrong; neurons, data flow, and mapping motifs were the natural starting point before InceptionV1 circuits broke for unexpected reasons.
More from Research
- SWAN: a semantic-level text watermark that survives paraphrasing — burkov · 2026-10-16
- Interp researchers push back: 'networks are already clean' claim rests on dropout artifact — aryaman2020 · 2026-10-16
- LayerTrim: training-free pruning keeps reasoning LLMs intact on AIME — SeoulNatlUniv · 2026-10-16
- Simular presents neuro-symbolic computer use agents at BayLearn — xwang_lk · 2026-10-16
- Google's Hartmut Neven disputes chip test, outlines plan to link a brain to a quantum computer — Chris_Armstrong · 2026-10-16
- T=16 flow model generates 16 frames per inference for smoother motion — pixlpa · 2026-10-16