Where-OPD: synthetic-scene spatial self-distillation boosts MLLM perception by 3.23 points
valeocorg · hf · 2026-10-02
The Where-OPD paper introduces spatially guided on-policy self-distillation for multimodal LLMs. Instead of human-annotated grounding data or external teachers, the EMA teacher receives textual spatial guidance on query-relevant visual elements, trained on procedurally generated synthetic scenes with free object identities and coordinates.
Key points:
- Teacher locates and integrates evidence across image regions; the student learns to reproduce this from image and question alone
- Post-training uses only synthetic scenes, yet gains transfer to real-world perception benchmarks
- +3.23 points average across CVBench, V, ZoomBench, BLINK, HR-Bench, and MME-RealWorld
Project page: github.com/sirkosophia/Where-OPD
More from Multimodal
- Inflect v3: open-source local TTS with a 157K-param model, Apache 2.0, 0.74% WER — pmttyji · 2026-11-17
- Seedance 2.5 demo shows Hollywood-level cinematic action realism — SimplyAnnisa · 2026-11-17
- RelayVSR open-source 4x video upscaler: 144p clip upscaled in ~2 min on a 4070 — Ant_6431 · 2026-11-17
- Suno pulls off noise/math-rock drums and atonal piano on request — AndyMasley · 2026-11-17
- Seedance2.5-generated Ramayana manga chapter: Jatayu's last stand — CurieuxExplorer · 2026-11-17
- Creator adapts the entire Ramayana into a manga using Higgsfield's Mangaka — CurieuxExplorer · 2026-11-17