New chain lens interpretability tool sparks rethink: LLMs may not be impenetrable black boxes
thesephist · x · 2026-11-21
- The author introduces chain lens, an interpretability tool developed over the past month: it augments existing token lenses (like the J-lens) with the model's multi-token prediction head, upgrading single-token readouts into multi-token phrase readouts. The idea is strikingly simple—just reuse the component literally named "multi-token."
- This prompts reflection on why we assume AI is inherently hard to understand: "black box trauma" from symbolic AI's death, "every time I fire a linguist performance goes up," the bitter lesson, XAI's failure, and early setbacks like the rarity of OpenAI's sentiment neuron and widespread superposition.
- Yet recently, new understanding avenues have surfaced in plain sight: J-lens found a simple logit lens improvement nobody noticed for six years, scalable interp NLAs and oracles are starting to work, and models are getting better at introspection and self-explaining their mechanics. The words may not be riding on an inscrutable ocean after all.
More from AGI Musings
- Mathematician Levent Alpöge went from proving theorems with Bhargava to working with Claude — burny_tech · 2026-11-21
- Reading old books matters: without them you're a hostage to the present — 0xkarasy · 2026-11-21
- Which job will be the last one AI can't replace? Reddit votes plumber — No-Childhood-113 · 2026-11-21
- Two years after mocking 'AI proved RH' claims, skeptic updates: now we do — burny_tech · 2026-11-21
- 'AI just solved math's hardest problem, but surely my mid-level accounting job is safe' — birchlse · 2026-11-21
- AI trainer's pivot: models now do 'most things', so the bottleneck is the idea — beglen · 2026-11-21