Shengshu's Vidu S2 edits live video streams in real time — swap outfits, faces and scenes
量子位 · wechat · 2026-09-15
Shengshu Technology released Vidu S2, just 69 days after S1 and fully available on day one. Three headline upgrades:
- S2-Editing: real-time outfit, character, background and style swaps on a playing video stream — live 'Photoshop for video.' Testing shows solid identity consistency and cloth deformation, with occasional smearing and hat-hair artifacts.
- S2-Avatar: real-time interactive avatars bumped from 540p to 720p, now executing large motions (dancing, turning) on voice command and accepting reference images mid-conversation to hold objects or change clothes.
- Real-time spatial video: streams can be converted to stereo views for VR headsets, treating the physical world as a canvas.
Tech: Self-Replay Forcing curbs error drift in streaming generation; a Backbone-Refiner two-stage architecture with async pipelining keeps 720p frame rates; redesigned positional encodings align reference-image injection with timestamps; a VLM agent tracks scene state to plan generation; DPO and Streaming NFT handle RL in the bidirectional and streaming stages. Led by Zhang Jintao, PhD student of Tsinghua's Zhu Jun; technical report available.
Related event: Shengshu releases Vidu S2: real-time avatars and streaming video editing(9 posts)→
More from Multimodal
- Pika's pika-h3-flex Space Trends on Hugging Face — Pika-Labs · 2026-11-02
- awa-video-reel: free Claude Code skill auto-edits shorts and records website B-roll — TawohAwa · 2026-11-02
- Hobbyist 3D-scans Yokohama's abandoned Negishi racecourse grandstand with 3DGS — janusch_patas · 2026-11-02
- Drone-captured 400-ft guyed tower rebuilt as 3DGS via Metashape and LichtFeld Studio — janusch_patas · 2026-11-02
- TBC launches brain-cell-inspired AI video model Mischief at $0.005 per second — danfaggella · 2026-11-02
- Even a skeptic almost fell for an AI fruit cartoon reel — menhguin · 2026-11-02