Researchers debate whether five years of mech interp work amounted to negative knowledge
jd_pressman · x · 2026-10-16
jdpressman debates with @tszzl and @LakeSnyde about the value of interpretability research: if "polysemanticity" turns out to be kind of fake, then paying attention to mech interp over the last five years may have given you negative knowledge compared to ignoring it entirely.
He adds that the same applied to AI broadly before GPT-2: the foundation-model paradigm of training on everything then tuning or prompting for specific tasks is genuinely unintuitive — EEG papers still don't get it.
Related event: Researchers Debate Counterintuitive Foundations of AI Paradigms(2 posts)→
More from Research
- AI will be superhuman at science regardless, and Europe can't opt out — teortaxesTex · 2026-10-16
- SERO augments long-horizon agentic RL with executable oracle continuations — Shanghai-AI-Laboratory · 2026-10-16
- Tsinghua and Amap propose OmniVerifier-C1, a graded verifier beating binary judges — Bowei Liu · 2026-10-16
- Alibaba open-sources MAI-Gym, an all-in-one GUI agent eval and RL training platform — Tongyi-MAI · 2026-10-16
- Peking University's SimForcing distills simulation motion priors into robot world models — PekingUniversity · 2026-10-16
- CoWorld Reasoner uses an evolving video world model as a visual reasoning engine — ModalityDance · 2026-10-16