VAD isolates visually supported corrections for multimodal distillation and beats direct teacher supervision on six benchmarks

Kangning Zhang · hf · 2026-08-04

What the paper does

The paper studies multimodal on-policy distillation (OPD), where a privileged-view teacher supervises student-generated trajectories.

Core idea: Visual Attribution Distillation (VAD)

Instead of treating teacher corrections as a single mixed signal, VAD tries to isolate the part that is actually attributable to visual evidence:

It evaluates the same fixed teacher with and without the relevant evidence.
The change in centered log-probabilities is used as a signed proxy for the evidence direction.
The original correction is projected onto that proxy to separate an intervention-aligned component from an unexplained residual.
Training then uses a student-anchored reconstructed target from the aligned component, with the privileged teacher only as a weak regularizer.

Results

Across six fine-grained visual benchmarks at 4B and 9B scales, VAD outperforms:

direct privileged-view distillation
visual-advantage weighting

Token-level and controlled-target analyses suggest the method better captures task-relevant visual corrections, especially when the evidence refutes a wrong answer.

Original post →

VAD isolates visually supported corrections for multimodal distillation and beats direct teacher supervision on six benchmarks

More from Multimodal