Alibaba's TaoLive-AIGC releases TaoMate-FVSR, audio-guided facial video restoration
TaoLiveAIGC · hf · 2026-11-03
- Alibaba's TaoLive-AIGC team released TaoMate-FVSR on arXiv, an audio-guided facial video restoration framework.
- Target problem: AI-generated videos often render detailed scenes but leave small facial regions blurry; purely visual super-resolution can't ensure recovered lips and teeth match the spoken audio.
- Method: restoration inside a reference-guided video flow model, with source-noise mixing initializing from partially preserved observations and structured reference injection separating appearance anchors from continuous motion context.
- Training combines degraded image/video restoration with correction of degraded dubbed sources under target audio, supporting visual dubbing when audio is swapped.
- Outperforms prior methods on talking-head restoration benchmarks.
More from Multimodal
- ComfyUI Workflow Generates 1080p Video with MiniMax H3 on Only 6GB VRAM — cgpixel23 · 2026-11-03
- Midjourney Recipe: 5 Weighted --sref Styles for Cinematic Light and Motion — michaelrabone · 2026-11-03
- LTX 2.5 users struggle to lock static cameras even with the Cinemagraph LoRA — bytebitter33 · 2026-11-03
- Tiny latent diffusion model trained inside LLMs generates MNIST digits, 76% recognized — zeeshanp_ · 2026-11-03
- Realism LoKr for Krea 2 trained on 749 images, blind voting arena picks the best epoch — Wide_Director_8897 · 2026-11-03
- Stunning AI-generated dieselpunk elf warrior image showcases detailed mechanical design — azed_ai · 2026-11-03