vgel team explores Loom Lens: sampling MTP heads like a language model
The vgel team introduced 'Loom Lens,' tree-reading hidden states by sampling MTP heads like a language model. The full write-up includes complete results, though the method has not yet significantly beaten the baseline.
2026-11-12 ~ 2026-11-12 · 2 related posts
- Episode 1: Chain Lens: Training-Free Multi-Token Readout from Model Activations(2026-11-12, 6 posts)
- Episode 2: All 1.1M MLP Neurons of Qwen 27B Labeled for Just $5(2026-11-12, 2 posts)
- Episode 3: vgel team explores Loom Lens: sampling MTP heads like a language model(2026-11-12, 2 posts)
- Loom lens: sampling the MTP head like a language model to get tree-shaped activation readouts — voooooogel · 2026-11-12
- Loom Lens Experiment: Parallel Multi-Line MTP Sampling Shows Potential for Interp Agents — voooooogel · 2026-11-12