Tencent's SyncReward builds human-aligned audio-visual sync reward from 80K ratings
tencent · hf · 2026-10-15
Tencent WeChat and Renmin University release SyncReward, addressing temporal misalignment between generated audio and visuals. They build SyncReward-Data-80K (80K professional sync ratings across 8 audio-visual generators, covering Speech/Onset/Instrument/Ambient events) and SyncReward-Bench (1K clips, multi-rater). Evaluation shows existing sync metrics and omni-modal models misalign substantially with human judgment. SyncReward combines semantic-temporal contrastive pretraining on real clips with human supervision on generated clips: strongest agreement with humans as a metric, and as a reward it improves sync via best-of-N selection and RL post-training. Data, benchmark, model, and code will be open-sourced.
More from Multimodal
- User updates AI-generated photos of a woman, claiming they're 100% synthetic — IndraVahan · 2026-10-15
- A detailed prompt for a 15-second one-take cinematic volcano disaster video — umesh_ai · 2026-10-15
- Physics benchmark scores 8 video models: top Seedance 2.5 only 57.76/100 — jiqizhixin · 2026-10-15
- H3 Turbo Neat LoRA delivers more realistic 4-step renders in ComfyUI for images and video — Ok_Tale7582 · 2026-10-15
- Spooky season: painterly AI Halloween illustrations with film-poster vibes — egeberkina · 2026-10-15
- Classic artworks reimagined as animated pixel art with Gemini 4 — fofrAI · 2026-10-15