UniAVC unifies audio-video generation via progressive training and asymmetric noise scheduling
Kien T. Pham · hf · 2026-10-20
HKUST and Huawei Research present UniAVC, a single model for text-conditioned joint audio-video synthesis, bidirectional cross-modal generation, and instruction-guided editing.
- Core idea: train separate audio and video branches on abundant unimodal data first, then align with scarce paired data
- Identifies two flaws of prior work: shared noise schedules ignore information-density disparity (video needs prolonged high noise for spatial structure, audio settles earlier), and lockstep denoising leaves conditioning workflows untrained
- Uses independent noise schedules and asymmetric supervision for joint synthesis and editing
- Project page: hkust-longgroup.github.io/UniAVC
More from Multimodal
- Feeding the whole Animator's Survival Kit to Opus 5.5 yields decent keyframe animation — MoonL88537 · 2026-10-20
- Glimpse launches as a visual thinking model claiming Astra-level reasoning benchmarks — LinusEkenstam · 2026-10-20
- ComfyUI lets you pipe images into an LLM — how do you do the same with audio? — PhilosopherSweaty826 · 2026-10-20
- AI-generated mecha battle video looks straight out of a movie — Key_Layer6250 · 2026-10-20
- Free ComfyUI templates chain local LLM lyrics into YuE2 song rendering, runs on 12GB VRAM — Vivid_Promise1700 · 2026-10-20
- DeepSeek 4.1 Flash builds video animation in 12 min for $0.20, self-scores 7.5/10 — oran_ge · 2026-10-20