Chain Lens: Training-Free Multi-Token Hidden-State Readout by Reusing MTP Heads
voooooogel · x · 2026-11-12
voooooogel published a LessWrong post introducing the chain lens, a new interpretability technique that extends single-token lenses like the J-lens into multi-token readouts without any training.
The trick: OSS models already ship MTP (multi-token prediction) heads that take a hidden state and predict future tokens, normally for speculative decoding. The authors feed the J-lens-transported hidden state into the MTP head and sample from it like a language model to roll out a token sequence verbalizing the state. Refinements such as contrastive decoding against a generic hidden-state prior further improve results.
Related event: Chain Lens: Training-Free Multi-Token Readout from Model Activations(6 posts)→
More from Research
- Intology Founder at YC Paper Club: Drug Failures Stem From Picking Wrong Targets, Not Bad Molecules — yusufroohani · 2026-11-12
- DeepMind's FlipTest benchmark: best video world models score below random guessing — AlexGDimakis · 2026-11-12
- AI Predicts Tooth Decay Risk from Brazilian Survey Data for Public Health Planning — Brighter-Side-News · 2026-11-12
- Meta FAIR open-sources e-means: 10–45x faster k-means with lower loss than Lloyd's algorithm — generativist · 2026-11-12
- Google and Artificial Analysis launch MKA benchmark to verify agent deliverables — OnlineInference · 2026-11-12
- AI models can prove anything, making 'mathematically trivial' ICLR reviews obsolete — nanjiang_cs · 2026-11-12