Self-Supervised Visual On-Policy Distillation
Yijiang Li, Yijun Liang, Yunjie Tian, Bingyang Wang, Ke Zhang, Zhenfei Yin, Di Fu, Philip Torr, Nuno Vasconcelos
cs.CV, cs.AI
2026-08-14
S2VOPD creates teacher-student asymmetry by degrading only the student's view, lifting Qwen3.5-4B's average accuracy on six perception benchmarks from 70.7% to 77.4%, above Qwen3-VL-235B.
On-policy distillation for vision-language models depends on the teacher knowing something the student doesn't. Usually that means a bigger, stronger teacher model or from privileged supervision like ground-truth answers or region annotations. Most real settings have neither. This paper asks where a useful teacher-student gap can come from when nothing privileged is available at all.
The trick is to invert who gets less information. Instead of giving the teacher more, the paper takes information away from the student. The teacher is an EMA (exponential moving average) copy of the same model, and it sees the clean image; the student sees a strongly augmented, degraded version of the same image. Both roll out on the same question and generate the same trajectories, and the teacher's next-token distribution on the clean image is distilled into the student via a generalized Jensen-Shannon divergence. The student's own blindness creates a predictive gap functionally equivalent to a better-informed external teacher, with no labels, no rewards, and no separately trained stronger model needed.
The paper runs a systematic sweep over four augmentation families: information reduction (downscaling, noise), geometric (rotation, cropping), photometric (brightness/contrast/saturation), and occlusion (random erasing, GridMask). It finds three patterns: asymmetry itself helps (all four families beat the base model, while symmetric self-distillation without any gap actually hurts); strength has a sweet spot (performance rises then falls as the induced gap grows); and the gap must stay task-consistent. Cropping can produce a huge teacher-student gap, but once it removes the visual evidence needed to answer the question, that gap becomes noise rather than signal. The winning recipe composes two operators: downscale to 0.3-0.6x resolution, then add Gaussian noise.
| Setup | V | HR-8K | MME-RW | Avg (6 benchmarks) |
| Qwen3.5-4B base | 84.29 | 63.86 | 63.86 | 70.68 |
| S2VOPD (4B) | 87.43 | 83.62 | 72.87 | 77.44 |
| Qwen3-VL-Instruct-235B | 91.10 | 85.50 | 74.82 | 75.75 |
| GPT-5.4 | 76.96 | 84.00 | 71.87 | 72.78 |
S2VOPD lifts the 4B model's average across six fine-grained perception benchmarks from 70.68% to 77.44%, beating Qwen3-VL-235B (75.75%, 59x more parameters) and GPT-5.4 (72.78%). On the same Vision-OPD-6K training data, methods with privileged information, ZwZ (ground-truth regions) and OPSD (ground-truth answers), trade perception gains for math-reasoning damage: ZwZ drops MathVerse by 27.1 points, OPSD drops MathVision by 9.3 points. S2VOPD doesn't have that tradeoff. An ablation removing the student-side augmentation (teacher and student see the same image) collapses the gain back to base-model level (70.52%), confirming the improvement comes almost entirely from the constructed asymmetry rather than from the EMA teacher improving on its own (a frozen teacher still recovers 93% of the gain).
For teams doing VLM post-training without access to a stronger teacher or labeled data, this is a route to improving small-model perception without either. The transferable idea isn't the specific recipe (downscale + noise) but the design principle: subtract information from the student side and tune augmentation strength against the induced KL/JS gap. The paper gives a usable strength-vs-performance curve (peaking near a JS divergence of about 0.014) as a diagnostic for tuning.
The paper itself shows cropping breaks spatially-grounded questions, meaning the method is sensitive to which augmentations are chosen — a different task family (spatial reasoning instead of fine-grained perception) could need a completely different optimal augmentation set. Evaluation is limited to perception benchmarks plus three math-reasoning benchmarks; it's untested on tasks needing longer-horizon planning or multi-turn interaction. And the approach fundamentally requires a task where the input can be meaningfully "degraded" — how it would apply to modalities without an obvious degradation axis (pure text reasoning, for instance) is left open.