Ant Group and Westlake Heart Spur open-source Duo v1, a bidirectional world model with real-time interactive agents
智东西 · wechat · 2026-11-18
- Ant Group's research institute and Westlake Heart Spur released and open-sourced Duo v1, a reciprocal world model where virtual agents natively participate in real-time audio/video interaction instead of just simulating physical scenes.
- Architecture: an Intent module decides every 0.7s chunk whether to speak or wait and passes internal hidden states to a Realization module that streams matching audio and video, with unified-timeline causal masking keeping generation aligned.
- Training spans three stages (Intent causal participation, frozen-Intent joint audiovisual generation, distribution-matching distillation into a 4-step generator), plus a fixed first-segment reference frame to curb quality degradation over 60-second generations.
- Human evaluation across 30 real interactions (180 checks) hit 93.3% pass rate, with 100% on language adherence, speech intelligibility and visual identity stability; content responsiveness (73.3%) remains the weak spot.
- Tech report, training code and model weights are all open-sourced.
Related event: Ant Group and Xixinchent Open-Source Reciprocal World Model Duo(2 posts)→
More from Multimodal
- LMMS-Lab: codec sampling beats uniform frame sampling on general video tasks — liuziwei7 · 2026-11-18
- AI-generated Robert Pattinson photo set is nearly indistinguishable from real photography — justin_hart · 2026-11-18
- From 1000-step DDPM to 1-step generation: a hands-on turbo diffusion walkthrough — RisingSayak · 2026-11-18
- Clueso v2: AI video editor lets teammates pin comments to the second and the AI makes the edit — aakashgupta · 2026-11-18
- Users seek a simple Director node for Minimax H3 workflows — Kind_Owl2245 · 2026-11-18
- Adobe Firefly ambassador shares full workflow for embroidery-style AI animated short — azed_ai · 2026-11-18