Tongyi Launches Wan-Streamer v0.2 Real-Time AV Model

通义实验室 · wechat · 2026-07-17

Tongyi Lab released Wan-Streamer v0.2, focusing on real-time full-duplex omni-modal understanding and generation. It aims to unify listening, seeing, speaking, and acting within a single end-to-end Transformer.

关键指标

End-to-end interaction latency of 550ms (200ms model latency, 350ms network)
Output resolution increased from v0.1's 192×336 to 640×368@25FPS
Supports real-time understanding and synchronized generation of text, audio, and video

架构变化

Replaces the traditional "ASR → LLM → TTS → Animation" pipeline with a unified CausalTimeline
Adopts a StreamingUnit closed loop approx every 160ms: perceive input, update state, generate audio-visual latents, and decode output
Uses a Thinker-Performer dual-pathway to split low-latency inference from high-res video generation
Thinker runs on a single GPU for real-time perception, state updates, and audio decoding; Performer leverages multi-GPU Ulysses parallel processing for high-res video latents
Temporal overlapping decouples the extra computational cost of high graphics from the latency-sensitive path

应用场景

Video-call AI assistants: spoken language practice, interview simulations, psychological counseling
Contextual companionship and education: assessing comprehension via facial expressions, cooking guidance
Immersive game NPCs
Accessibility: generating more natural audio-visual responses for hearing/visually impaired users

The end of the post provides the project repository and links to the v0.1, v0.2, and v0.3 papers.

Original post →

Tongyi Launches Wan-Streamer v0.2 Real-Time AV Model

More from Multimodal