Northeastern researchers find 'concept induction heads' that copy word meanings, not just tokens
davidbau · x · 2026-09-30
A Northeastern team (Sheridan Feucht, Eric Todd, Byron Wallace, David Bau) released "The Dual-Route Model of Induction", distinguishing two kinds of induction heads in LLMs: token induction heads that copy bit-by-bit (as Olsson et al. 2022 showed) and concept induction heads that copy word meanings at the semantic level. Working together, they form a dual-route model of induction.
- The design is inspired by psychology's dual-route model of human reading: familiar words are processed whole via a lexical pathway, while unknown words are decoded letter-by-letter.
- The paper ships with ArXiv full text, GitHub code, a Colab demo, and slides; a related "Concept Lens" technique and a vector-arithmetic bonus paper are also available.
- Author David Bau notes these methods can summarize attention-head-bundle OV transforms, yielding readouts better than the logit lens in VLMs.
More from Research
- MIT's Ataraxo AI beats top Stratego players with self-play and decision-time planning — nordicinst · 2026-09-30
- New Research: AI as Tutor Beats AI as Substitute — and No AI — CackleRooster · 2026-09-30
- McKinsey: AI-enabled drug candidates cut discovery time by ~15-80% — HealthcareAIGuy · 2026-09-30
- Five good results won't tell you if your AI model works: a five-slot validation test — bravo_abad · 2026-09-30
- AI model originates new proof of Odlyzko-Poonen conjecture, fully formalized in Lean — fedzbar · 2026-09-30
- Hillel Wayne: TLA+ is great, but 'formal methods will save AI' hype misses what it can't even express — leland_mcinnes · 2026-09-30