Interpretability debate: cats and grandmasters show understanding needn't be human-readable
davidmanheim · x · 2026-11-05
David Manheim argues that like humans who recognize cats without a GOFAI-style method, models may complete key tasks without any human-interpretable core of understanding — much of it may be inherently high-dimensional.
More from AGI Musings
- Blogger proposes scrapping ML conference checklists now that everyone uses LLMs to vet papers — bremen79 · 2026-11-05
- Critique: adversarial safety plus blind obedience training breeds models that silently act on doubts — repligate · 2026-11-05
- Paras Chopra: Future research will decouple solving (AI) from understanding (humans) — paraschopra · 2026-11-05
- Game developers push back on AI backlash: 'AI is not a category, it is simply a tool' — TinfoilTricorn · 2026-11-05
- JeffLadish: score-maximizing ASI will take over — Yudkowskian case restated — JeffLadish · 2026-11-05
- JeffLadish: models aren't aligned by default — they care about scores, not us — JeffLadish · 2026-11-05