Emory's DLR makes VLMs re-look at images every reasoning step, beating baselines on MathVista

新智元 · wechat · 2026-09-26

A Decompose-Look-and-Reason (DLR) framework from Emory University, accepted to EMNLP 2026, tackles the problem where multimodal chain-of-thought grows longer in text while visual evidence fades.

Method: each step generates a premise (what visual fact to verify), a visual grounder extracts premise-conditioned continuous latents via cross-attention instead of fixed ROIs, and the VLM generates a grounded rationale before iterating.

Training: three stages—grounder pretraining with bidirectional InfoNCE, structured SFT, then RL extended to continuous latents via a Spherical Gaussian Latent Policy that perturbs along semantic directions on the hypersphere, jointly optimized with the text policy using Dr.GRPO.

Results: on Qwen3-VL-8B-Thinking, DLR scores 83.8 (V), 82.7 (MathVista), 63.5 (MMMU-Pro), 76.2 (MMStar), gaining +4.2/+3.5/+2.6/+3.9 points over the backbone. Removing the latent policy objective drops MathVista from 82.7 to 57.1. Occluding top-attention regions cuts V accuracy by 47.3 points; case studies show the baseline burning 15K tokens on wrong answers while DLR resolves them stepwise.

Original post →

Emory's DLR makes VLMs re-look at images every reasoning step, beating baselines on MathVista

More from Multimodal