Evidence-RL: Eliminating VLM Shortcuts via Counterfactual Causal Intervention
Haojie Huang · hf · 2026-08-11
Current Vision-Language Models (VLMs) often rely on language priors or dataset shortcuts. This paper proposes Counterfactual Evidence Disentanglement (CED) to audit VLM grounding during training.
- Mechanism: It neutralizes object-centric evidence regions and compares the resulting support drop to test causal dependence on local visual evidence.
- Results: Combined with GRPO, this signal rewards correct answers relying on true evidence rather than shortcuts. It outperforms prior RL-based post-training methods across 9 benchmarks and 4 backbones without adding inference-time overhead.
More from Research
- Can LLMs Make Scientific Leaps? Paper Argues Structural Limits — ZhiyuChen4 · 2026-08-11
- Mathematicians Leverage AI to Solve Another FrontierMath Problem — alphacolony21 · 2026-08-11
- HAROOD Selected as KDD 2026 Oral: Advancing Out-of-Distribution Detection — jindong_wang92 · 2026-08-11
- Robotics VLA Paper 'OAT' Named RSS 2026 Outstanding Paper Finalist — Haoyu_Xiong_ · 2026-08-11
- MIT's DAAAM Uses VLMs to Build Real-Time 3D Dynamic Scene Graphs — tom_doerr · 2026-08-11
- UAlberta RLAI Talk: Viewing Reinforcement Learning Through a Physics Perspective — MarlosCMachado · 2026-08-11