Debate: neural networks are built to be read — is mech interp easier than claimed?
basedjensen · x · 2026-10-17
tszzl argues that "everything in the neural network was built in some sense to be read" by downstream readers, calling this a scientific revolution. basedjensen adds that instrumenting networks should be easy, questioning where the mech interp community's prior that you can't instrument networks came from. The exchange touches a core methodological split in interpretability research.
More from AGI Musings
- Dev scrapes all 43,013 ICLR 2027 submissions and builds a citation-ranking site with Codex and Claude — yshan2u · 2026-10-17
- Scobleizer visits TheHumanoidHub: robots for devs this year, everyone else after 2030 — Scobleizer · 2026-10-17
- The Future of Media: People With Direction vs. People Generating Noise — AIandDesign · 2026-10-17
- Tuhin: AI Is Dangerously Concentrating Power in the Hands of a Few — joonasvirtanen · 2026-10-17
- The simulation argument maps neatly onto the Sleeping Beauty problem — AaronBergman18 · 2026-10-17
- Engels' Pause and AI: why efficiency gains don't equal value creation — oran_ge · 2026-10-17