VCSD boosts Qwen3-VL on ViRL39K without external teachers or evidence
UMCP · hf · 2026-07-24
What it is
A new method called Visual Contrastive Self-Distillation (VCSD) removes the need for an external teacher or privileged visual signals in on-policy self-distillation for vision-language models.
How it works
- At each student-generated prefix, an EMA teacher is run twice under the same prompt/prefix:
- once with the original image
- once with a content-erased control image
- The token-level log-probability gap highlights tokens specifically supported by the visual content.
- VCSD sharpens the teacher’s original-image distribution within plausible support, then distills that distribution into the student.
Results
On ViRL39K, VCSD consistently beats matched OPSD across Qwen3-VL and Qwen3.5 variants.
- Qwen3-VL 2B: 62.27% → 67.04%
- Qwen3-VL 4B: 71.30% → 73.16%
- Qwen3-VL 8B: 72.51% → 76.26%
Why it matters
The method needs only the image and question, adds no inference-time cost, and avoids external teachers, answers, evidence signals, or reasoning traces.
More from Multimodal
- Game-native world model adds 108M-frame dataset and explicit state supervision — Scobleizer · 2026-07-24
- A creator built a 4-minute animated short with FLUX.1 Dev and LTX-Video 2.3 — developervkmp · 2026-07-24
- A new pixel-space benchmark says image generators can reason spatially, but text VLMs still lead — OmniAI-ZJU · 2026-07-24
- Higgsfield’s MCP turns Claude into a no-code video studio with one URL — socialwithaayan · 2026-07-24
- A Claude plus video-model workflow can now turn one sentence into a 10-minute documentary — PrajwalTomar_ · 2026-07-24
- New Midjourney style shared with exact prompt settings and example images — azed_ai · 2026-07-24