VCSD boosts Qwen3-VL on ViRL39K without external teachers or evidence
UMCP · hf · 2026-07-24
What it is
A new method called Visual Contrastive Self-Distillation (VCSD) removes the need for an external teacher or privileged visual signals in on-policy self-distillation for vision-language models.
How it works
- At each student-generated prefix, an EMA teacher is run twice under the same prompt/prefix:
- once with the original image
- once with a content-erased control image
- The token-level log-probability gap highlights tokens specifically supported by the visual content.
- VCSD sharpens the teacher’s original-image distribution within plausible support, then distills that distribution into the student.
Results
On ViRL39K, VCSD consistently beats matched OPSD across Qwen3-VL and Qwen3.5 variants.
- Qwen3-VL 2B: 62.27% → 67.04%
- Qwen3-VL 4B: 71.30% → 73.16%
- Qwen3-VL 8B: 72.51% → 76.26%
Why it matters
The method needs only the image and question, adds no inference-time cost, and avoids external teachers, answers, evidence signals, or reasoning traces.
More from Multimodal
- Adobe Firefly video editor walkthrough: one-sentence Quick Cut turns 32s into 20s — LudovicCreator · 2026-10-26
- Hi3D V1.0 open-sourced with 2M training assets and 90% slicing success rate — kimmonismus · 2026-10-26
- Claude Can Replace Your YouTube Thumbnail Designer in 10 Minutes — thedankoe · 2026-10-26
- Multimodal AI Is Not Useless: A Practitioner Pushes Back on Growing Skepticism — thoma_gu · 2026-10-26
- One prompt, full Blender animation: dev says he never touched Blender once — iamfakhrealam · 2026-10-26
- Dense instance segmentation via Muse Wonder + SAM 3.1 on Meta's Model API — nikhilaravi · 2026-10-26