Evidence-RL: Towards Evidence-intensive Visual Reasoning
Haojie Huang, Xinlei Yu, Chengming Xu, Zhangquan Chen, Cheng Yang, Qingdong He, Yu Yang, Jiangning Zhang, Xiaobin Hu
cs.CV, cs.AI
2026-08-08
Evidence-RL masks a proposed image region at train time and rewards answers whose likelihood drops when that evidence is removed. Gains hold across 9 benchmarks and 4 backbones.
Vision-language models (VLMs) often get the right answer without actually looking at the image. Ask how many wheels a car has and the model may never glance at the picture, just parrot "a sedan has four wheels." This shortcut-hunting shows up most on counting, spatial reasoning, and fine-grained recognition, where language priors and dataset statistics carry the answer.
Existing perception-aware post-training tries to fix this with global image perturbations or attention proxies that nudge the model to "use the image." They share a blind spot: none of them can check whether a specific answer depends on a specific region of the image. A single answer can be supported by three causal paths at once: real evidence, irrelevant context, or language priors. Correctness alone cannot tell them apart. Evidence-RL drags "why the answer is right" into the training signal.
The core mechanism is Counterfactual Evidence Disentanglement (CED), a counterfactual audit run once per candidate answer:
The non-evidence regions act as a local null hypothesis. If an answer is equally sensitive to masking any region, what it tracks is the act of masking itself, not real evidence. Only when the evidence region causes a larger drop than the references does m go high.
The margin feeds into the GRPO reward: R = answer-correctness x g(m) + epsilon x m. Correctness still drives the main gradient, but the gate g(m) separates rollouts that are all correct, and the one relying more on evidence scores higher. The paper's clean example is a car-wheel counting question. Two rollouts both answer "4." One leans on the "sedan = 2+2" prior and skips the image; the other counts "1 front + 2 rear + 1 spare." The CED gate hands the grounded rollout a 7x reward (0.78 vs 0.11), steering GRPO toward the trajectory that actually looked. The audit runs only at training time; inference has zero overhead.
Main results on Qwen2.5-VL-7B across nine benchmarks (CountBench, SpatialEval, HallusionBench, VLMsAreBlind, FREAK for grounding; MathVista, MMBench, MMMU, ScienceQA for general reasoning):
| Method | Avg | Delta vs base |
| Qwen2.5-VL-7B base | 61.19 | n/a |
| VAPO-Thinker | 60.05 | -1.14 |
| VPPO | 63.47 | +2.28 |
| PAPO-G-H | 63.80 | +2.61 |
| SophiaVL-R1 | 63.99 | +2.80 |
| Evidence-RL | 67.13 | +5.94 |
Evidence-RL posts the highest average and the largest gain, with no regression on any of the nine benchmarks. The most vision-heavy benchmarks move the most: VLMsAreBlind +7.58, FREAK +14.22, MMMU +6.84.
Across four backbones (Qwen2.5-VL-3B/7B, Qwen3-VL-8B, Qwen3.5-9B) the mean gains are +4.59, +5.94, +1.33, +11.34, and all 36 benchmark-by-backbone cells are non-negative.
The controlled ablation is the most telling test, same Qwen3.5-9B, same data and compute: correctness-only reward averages -1.24; retrained VPPO -1.78; retrained PAPO collapses to -21.39 (its global-perturbation KL objective traps the model in repetition loops, 25% repeated-segment rate); Answer-CED reaches +11.34. CED alone contributes +12.58. The same data and compute translate into transferable gains only through CED.
The VLM field has a quiet chronic problem: right answer, wrong reason. The model guesses correctly, you assume it read the image, but it was reciting a prior. The moment the distribution throws a counter-prior image at it (a three-wheeled car), it breaks. CED is the first to turn "does this answer causally depend on the right visual region" into a computable signal that fits inside an RL reward.
Practically it is light: audit only at training, zero inference overhead, no per-question evidence labels, just off-the-shelf COCO boxes. The paper also proves a structural point: any reward that observes only the answer text, no matter how strong the judge, cannot separate "grounded correct" from "prior-lucky correct," because both paths terminate at the same answer node and only an image-conditional signal breaks the tie. For anyone doing VLM post-training, this is a cleaner grounding signal than global perturbation or attention proxies.
The evidence region rests on weak COCO object priors with no per-question annotation. The authors concede this may miss attribute-level or relational evidence; the method is validated mostly on object-level interventions. ScienceQA gains are modest partly because that benchmark contains blank-image items, for example a 448x448 placeholder with "what is the mass of a cement truck." The base model answers from prior and gets credit; the CED-trained model says "cannot determine from image" and is marked wrong. That is CED behaving as designed, but it drags down the surface score. Mean replacement dampens but does not eliminate intervention artifacts. The CoT variant of CED is gameable, since the model can shorten its reasoning chain to raise the per-token average, so the default is the Answer variant that scores only the final answer span.