Peking University's Motion-Omni generates speech and full-body motion in sync, faster than real-time
aigclink · x · 2026-09-07
Peking University released Motion-Omni, an end-to-end spoken dialogue system that generates speech together with synchronized full-body motion (facial expressions, hands, upper and lower body).
- Motion is output natively from the internal hidden states of speech generation, so motion and audio are inherently synchronized — not speech-then-animation
- RTF 0.78, total response time 4.32s, faster than real-time; 5.4x faster than the LOM teacher cascading approach on the same audio
- Limitation: currently offline response generation (waits for the full user utterance), not streaming real-time
A step toward embodied AI interaction for digital humans, VTubers, and personal avatars.
Related event: PKU Releases Motion-Omni for Real-Time Speech-Motion Generation(3 posts)→
More from Multimodal
- Wan2GP lands on Pinokio: 1-click AI video on 6GB GPUs — cocktailpeanut · 2026-10-27
- WanGP v18.81 adds Tencent's Hy-Nomas world model, playable on 8GB VRAM — cocktailpeanut · 2026-10-27
- World models matter even without realtime: closed loops and persistent memory — cocktailpeanut · 2026-10-27
- Butter.video wins over skeptical creator with hundreds of tasteful timeline 'blocks' for video effects — beechinour · 2026-10-27
- AI Video Pits Brock Lesnar Against Duncan the Tall in a Toy Showdown — KloneStudios · 2026-10-27
- $50, 2-hour Kurzgesagt-style AI video reignites debate over intent in AI creation — motionbynick · 2026-10-27