David Bau: OV-transform summaries beat logit lens readouts in VLMs almost by accident
davidbau · x · 2026-09-30
Following up on the concept induction head research, davidbau highlights that students have been experimenting with "Jacobian lenses", and points to Sheridan Feucht's technique for summarizing attention-head-bundle OV transforms. Applied to VLMs, it produces readouts much better than the logit lens almost by accident.
More from Research
- Diffusion Models Tutorial Accepted to NeurIPS 2026 Alongside 7 Paper Acceptances — mittu1204 · 2026-09-30
- AMB3R-SLAM: Kilometer-Scale Real-Time SLAM on One Consumer GPU, Cutting ATE by 70% — rsasaki0109 · 2026-09-30
- Prefix-Reuse FLOPs: new metric exposes hidden cost of arbitrary context edits in LLM serving — RulinShao · 2026-09-30
- Tencent Hunyuan releases ExplorationBench to measure how AI systems explore — TencentHunyuan · 2026-09-30
- Fully open MolmoAct 2 tops independent robotics benchmark LIBERO-MAX on dynamic robustness — DJiafei · 2026-09-30
- CompVis improves Distributional Diffusion Models: 4.48 FID at 4 steps on ImageNet — CompVis · 2026-09-30