Peking U & Kling Team Release RefCaptioner, Tackling Video Understanding Hallucinations
jiqizhixin · x · 2026-08-19
Video models often hallucinate, e.g., confusing a woman's red dress with a distractor dog. Researchers from Peking University and the Kling Team introduce RefCaptioner, designed to ground visual phrases to specific reference images accurately. Their two-stage recipe combines supervised fine-tuning on mixed data with a custom RL step (HCD-GRPO) that rewards correct phrase-image binding and penalizes hallucinations. Trained on 20K videos and 171K reference images, the model achieves SOTA on the new MRVBench, outperforms open-source rivals, and produces human-preferred captions along with more faithful video generation.
More from Multimodal
- A deep dive into audio watermarking: hidden marks, ownership and monetization for AI music — TheChuckTone · 2026-08-19
- MiniMax H3 generates stunningly accurate "animals squeezing into jars" videos — doublescale · 2026-08-19
- Meta releases Online-3DGS-Monocular for real-time 3D reconstruction — rsasaki0109 · 2026-08-19
- MiniMax Launches Desktop Creation Platform with End-to-End Agent Workflows — aziz4ai · 2026-08-19
- Hands-on with MiniMax Design: Generating Egyptian-style music video from audio — aziz4ai · 2026-08-19
- AI-Generated Product Commercial: 'Freshly Made' Creative Showcase — littleteckmonkey · 2026-08-19