Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes
Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc, Nicolas Thome, Spyros Gidaris
cs.CV, cs.AI, cs.CL, cs.LG
2026-10-02
A frozen teacher gets hints listing where targets sit; the student distills this on-policy over 3,000 synthetic counting scenes, gaining 3.23 points on real-world perception.
On-policy self-distillation (OPSD) post-trains a model by letting a frozen or EMA copy of it, armed with privileged information the student never sees, supervise the student token by token on the student's own sampled responses. Ported to multimodal LLMs, the privilege has so far always meant better pixels: Vision-OPD hands the teacher a crop around the answer, Imagine-OPD a zoomed view, RP-OPSD a higher resolution. These choices shine when a task rewards zooming into one region and fall apart elsewhere. On Qwen3.5-4B, Vision-OPD adds 8.90 points on V and 14.56 on ZoomBench while dropping CountQA by 10.73; counting needs every instance scattered across the frame, and a crop discards most of them. Such pipelines also rely on human-annotated grounding data or an external model to locate regions.
The paper flips the question: instead of telling the teacher what to look at, can the privilege simply say where the evidence sits?
All training data is procedurally generated. A 1024×1024 canvas gets 12 to 40 colored geometric shapes (7 shapes, 12 colors), and each question asks how many objects match a color-shape pair. Because the generator renders the scene, it already knows every object's coordinates, so the privileged hint comes straight from scene metadata through a fixed template: "Scanning for green crosses: found one near (377, 400), found one near (213, 805). Counting: 2 total." One sentence carries the target category, each instance's location, and the final count. No human annotation, no external teacher, 3,000 image-question pairs in total.
During training the student sees only the image and question and samples 8 responses per prompt; the frozen teacher sees the same inputs plus the hint and scores the student's prefixes. The loss is a generalized Jensen-Shannon divergence (β = 0.5), computed over the student's 100 most probable tokens at each position, with gradients flowing only through the student. At inference the hint is gone and the model is used exactly as before.
Counting is a deliberate task choice: it forces the model to find every matching instance across the whole frame, which is precisely the scan-locate-tally behavior the hint narrates. Attention maps after training cover all question-relevant regions on chart questions, where the base model and Vision-OPD each miss some. The cost is tiny: one epoch takes 1.4 hours on 4 H100s for the 4B model, 5.8 hours for the 9B.
Across 15 benchmarks and three models, Where-OPD posts the highest average in every case.
| Setting | Metric | Result |
| Qwen3.5-4B | 15-benchmark average | 73.86 → 78.31 (+4.45) |
| same base, Vision-OPD | 15-benchmark average | +0.53 |
| same base, OPD-V / S2VOPD | 15-benchmark average | +1.48 / +1.40 |
| Qwen3.5-4B | ChartQA / EvoChart | +7.20 / +10.11 |
| Qwen3.5-4B | MME-RealWorld / HallusionBench | +10.48 / +12.20 |
| three models | real-world perception suite average | +3.23 / +1.07 / +1.29 |
Every training image is geometric shapes, yet the gains land on real photographs: within MME-RealWorld, autonomous-driving perception rises 24.70 and remote sensing 7.83; V rises 5.76. This synthetic-to-real jump is the paper's strongest result.
The ablation (seven-benchmark average, base 69.63) separates the ingredients: SFT lands at 69.48, GRPO at 69.70, OPSD with no privileged information at 69.56, basically no progress. An answer-only hint reaches 71.27, coordinates without the final count 72.35, coordinates plus the count 72.94. Using cropped images as the privilege instead drops CountQA from 32.00 to 20.22. A frozen teacher also beats every tested EMA update rate; the authors' guess is that updating the teacher on synthetic images introduces domain shift.
Three thousand generated images, a couple of GPU-hours, zero annotation, and broad gains across 15 benchmarks is an unusually good trade in MLLM post-training. It also opens a new axis for OPSD design: privilege as a textual statement of where evidence lives, complementary to privilege as a better view. For practitioners it is a cheap, reproducible recipe, and the code is public.
Honest framing: this is an encouraging empirical finding more than a solved mechanism. Why counting geometric shapes transfers to chart understanding is supported only by qualitative attention maps, and the headline gain concentrates in the smallest model.
The gains are uneven across models: +4.45 on Qwen3.5-4B but +1.76 on Qwen3.5-9B and +1.01 on Qwen3-VL-4B, with no analysis of why. High-resolution visual-search benchmarks barely move (ZoomBench +0.87, HR-Bench 4K −0.63), exactly where crop-based methods win, so the two families look complementary rather than substitutable. The 9B model had to be trained with open-ended questions because multiple-choice elicited bare answers with no reasoning tokens to supervise, a sign the pipeline is sensitive to prompt format. Two main-table results average three seeds, but ablations and everything else are single runs, and 3,000 scenes happens to be the best tested data size. The training task is one template over one synthetic distribution; whether other tasks and hint templates work as well is untested.