Peking U & Kling Team Release RefCaptioner, Tackling Video Understanding Hallucinations
jiqizhixin · x · 2026-08-19
Video models often hallucinate, e.g., confusing a woman's red dress with a distractor dog. Researchers from Peking University and the Kling Team introduce RefCaptioner, designed to ground visual phrases to specific reference images accurately. Their two-stage recipe combines supervised fine-tuning on mixed data with a custom RL step (HCD-GRPO) that rewards correct phrase-image binding and penalizes hallucinations. Trained on 20K videos and 171K reference images, the model achieves SOTA on the new MRVBench, outperforms open-source rivals, and produces human-preferred captions along with more faithful video generation.
More from Multimodal
- MLP-Splatting (ECCV 2026) code released: MLPs as light-field primitives instead of Gaussians — AjdDavison · 2026-10-16
- Digital human Fae codes her own facial expressions frame-by-frame with Claude Opus 5.5 and FACS — repligate · 2026-10-16
- Meta's Omnilingual ASR, covering 1,600 languages, named a TIME Best Invention of 2026 — AIatMeta · 2026-10-16
- ATAS: Zhejiang-led team builds world-state memory framework for consistent agentic video generation — CSProfKGD · 2026-10-16
- Flux 3 Klein's Default Templates Are Broken Without Its Prompt Upscaler — StrictMove5839 · 2026-10-16
- Kroma Touted as the Most Well-Rounded Image Model Yet on Civitai — Different_Fix_2217 · 2026-10-16