Tongyi Launches Wan-Streamer v0.2 Real-Time AV Model
通义实验室 · wechat · 2026-07-17
Tongyi Lab released Wan-Streamer v0.2, focusing on real-time full-duplex omni-modal understanding and generation. It aims to unify listening, seeing, speaking, and acting within a single end-to-end Transformer.
关键指标
- End-to-end interaction latency of 550ms (200ms model latency, 350ms network)
- Output resolution increased from v0.1's 192×336 to 640×368@25FPS
- Supports real-time understanding and synchronized generation of text, audio, and video
架构变化
- Replaces the traditional "ASR → LLM → TTS → Animation" pipeline with a unified CausalTimeline
- Adopts a StreamingUnit closed loop approx every 160ms: perceive input, update state, generate audio-visual latents, and decode output
- Uses a Thinker-Performer dual-pathway to split low-latency inference from high-res video generation
- Thinker runs on a single GPU for real-time perception, state updates, and audio decoding; Performer leverages multi-GPU Ulysses parallel processing for high-res video latents
- Temporal overlapping decouples the extra computational cost of high graphics from the latency-sensitive path
应用场景
- Video-call AI assistants: spoken language practice, interview simulations, psychological counseling
- Contextual companionship and education: assessing comprehension via facial expressions, cooking guidance
- Immersive game NPCs
- Accessibility: generating more natural audio-visual responses for hearing/visually impaired users
The end of the post provides the project repository and links to the v0.1, v0.2, and v0.3 papers.
More from Multimodal
- Magnific teases October 28 release: 'We're making it move' — cuenca · 2026-10-27
- Midjourney --sref demo: retro Americana Batman art series — gizakdag · 2026-10-27
- AI video is a new medium, not cinema or animation — and it's perfect for depicting dreams — Kyrannio · 2026-10-27
- Dad turns his toddler's 31-year-old quote into a song via SUNO kids' choir — NathanpmYoung · 2026-10-27
- Band's local-AI music video hits 200k views: 900 generations kept 64, 3-second clips beat face drift — Hwang0500 · 2026-10-27
- Adobe Firefly video editor walkthrough: one-sentence Quick Cut turns 32s into 20s — LudovicCreator · 2026-10-26