Interp researchers push back: new findings don't overturn core interpretability ideas
menhguin · x · 2026-10-16
- Context: livgorton worried a new paper would leave readers thinking a central interpretability idea rested on a mistake and has been overturned — she argues the evidence doesn't support that even if some findings prove valuable.
- menhguin called the paper a helpful list of personal intuitions, but says "recursive generalisation" isn't complete enough as a theory to declare anything solved.
- He notes multiple alternative framings: tokenisation as central to how models understand the world, attention as expressing conceptual understanding in context, earlier layers as syntax vs deeper layers — so the conclusions aren't conclusive.
- The exchange reflects interp researchers' caution: new findings can be valuable without being overturning.
More from Research
- Five pharma giants federally train OpenFold3 on 20,000 private structures, +11 pts — jiqizhixin · 2026-10-16
- SWAN: a semantic-level text watermark that survives paraphrasing — burkov · 2026-10-16
- Interp researchers push back: 'networks are already clean' claim rests on dropout artifact — aryaman2020 · 2026-10-16
- LayerTrim: training-free pruning keeps reasoning LLMs intact on AIME — SeoulNatlUniv · 2026-10-16
- Simular presents neuro-symbolic computer use agents at BayLearn — xwang_lk · 2026-10-16
- Google's Hartmut Neven disputes chip test, outlines plan to link a brain to a quantum computer — Chris_Armstrong · 2026-10-16