Tongyi Launches Wan-Streamer v0.2 Real-Time AV Model
通义实验室 · wechat · 2026-07-17
Tongyi Lab released Wan-Streamer v0.2, focusing on real-time full-duplex omni-modal understanding and generation. It aims to unify listening, seeing, speaking, and acting within a single end-to-end Transformer.
关键指标
- End-to-end interaction latency of 550ms (200ms model latency, 350ms network)
- Output resolution increased from v0.1's 192×336 to 640×368@25FPS
- Supports real-time understanding and synchronized generation of text, audio, and video
架构变化
- Replaces the traditional "ASR → LLM → TTS → Animation" pipeline with a unified CausalTimeline
- Adopts a StreamingUnit closed loop approx every 160ms: perceive input, update state, generate audio-visual latents, and decode output
- Uses a Thinker-Performer dual-pathway to split low-latency inference from high-res video generation
- Thinker runs on a single GPU for real-time perception, state updates, and audio decoding; Performer leverages multi-GPU Ulysses parallel processing for high-res video latents
- Temporal overlapping decouples the extra computational cost of high graphics from the latency-sensitive path
应用场景
- Video-call AI assistants: spoken language practice, interview simulations, psychological counseling
- Contextual companionship and education: assessing comprehension via facial expressions, cooking guidance
- Immersive game NPCs
- Accessibility: generating more natural audio-visual responses for hearing/visually impaired users
The end of the post provides the project repository and links to the v0.1, v0.2, and v0.3 papers.
More from Multimodal
- FLUX.2 Klein Drifts Hard on Character Expressions While Free Gemini Holds Likeness — wacomlover · 2026-09-11
- Tencent Hunyuan releases AuK code and weights on GitHub with ComfyUI and fine-tuning support — aigclink · 2026-09-11
- Tencent open-sources AuK, a unified 1.5B speech generation and editing model — aigclink · 2026-09-11
- Creator turns Bahamut vs Tiamat rivalry into an AI cinematic battle with Midjourney, GPT Image 2 and Seedance — azed_ai · 2026-09-11
- invideo launches AI agent-powered editor to automate repetitive editing tasks — azed_ai · 2026-09-11
- fable 5.1 recreates The Starry Night with 256,157 JavaScript brush strokes — cedric_chee · 2026-09-11