Tongyi Launches Wan-Streamer v0.2 Real-Time AV Model
通义实验室 · wechat · 2026-07-17
Tongyi Lab released Wan-Streamer v0.2, focusing on real-time full-duplex omni-modal understanding and generation. It aims to unify listening, seeing, speaking, and acting within a single end-to-end Transformer.
关键指标
- End-to-end interaction latency of 550ms (200ms model latency, 350ms network)
- Output resolution increased from v0.1's 192×336 to 640×368@25FPS
- Supports real-time understanding and synchronized generation of text, audio, and video
架构变化
- Replaces the traditional "ASR → LLM → TTS → Animation" pipeline with a unified CausalTimeline
- Adopts a StreamingUnit closed loop approx every 160ms: perceive input, update state, generate audio-visual latents, and decode output
- Uses a Thinker-Performer dual-pathway to split low-latency inference from high-res video generation
- Thinker runs on a single GPU for real-time perception, state updates, and audio decoding; Performer leverages multi-GPU Ulysses parallel processing for high-res video latents
- Temporal overlapping decouples the extra computational cost of high graphics from the latency-sensitive path
应用场景
- Video-call AI assistants: spoken language practice, interview simulations, psychological counseling
- Contextual companionship and education: assessing comprehension via facial expressions, cooking guidance
- Immersive game NPCs
- Accessibility: generating more natural audio-visual responses for hearing/visually impaired users
The end of the post provides the project repository and links to the v0.1, v0.2, and v0.3 papers.
More from Multimodal
- Gemini Omni Flash turns a boat cabin into a cave in Flow by Google — chrisfirst · 2026-07-22
- A simple workflow to turn a photo into an image prompt using Gemini, Grok, or GPT Image — harshitagu72595 · 2026-07-22
- A Reddit user proposes a consistency LoRA to keep anime and game scenes visually stable — ThirdWorldBoy21 · 2026-07-22
- Hand-painted figurines run through Seedance look eerily alive — cocktailpeanut · 2026-07-22
- An AI agent-made bayou country music video is making the rounds on Reddit — LazyKaleidoscope4696 · 2026-07-22
- Testing Qwen 3 Image: Map Borders Shift Based on Prompts, Includes Chinese Labels — NirantK · 2026-07-22