ShotStream: Real-Time Multi-Shot Video Generation

机器之心 · wechat · 2026-07-11

ShotStream: Real-Time Streaming Multi-Shot Video Generation

The CUHK MMLab and Kuaishou Kling team proposed ShotStream, a real-time streaming multi-shot generation framework designed for interactive long video creation. The authors point out that traditional methods for multi-shot long videos typically suffer from high latency and poor interactivity: users often have to submit the entire script and storyboard at once, and a single failed shot can ruin the whole generation process.

Key Innovations

Reformulating multi-shot synthesis as "next-shot generation based on historical context," allowing users to adjust the narrative in real-time via dynamic streaming prompts during generation.
Dual-cache memory mechanism:
GlobalCache ensures cross-shot consistency;
LocalCache maintains fluidity within a single shot.
To avoid temporal ambiguity caused by the dual cache, the authors introduced RoPE residual/discontinuous position encoding to distinguish between caches.
Proposed a two-stage Self-Forcing distillation strategy to mitigate error accumulation in long video autoregressive generation.

Performance & Implementation

Achieved 16 FPS inference speed on a single H200 GPU.
Improved generation efficiency by more than 25x compared to bidirectional models.
The paper has been accepted by ECCV 2026.
Training, testing code, and models are open-sourced.

The article also adds experimental conclusions: ShotStream outperforms or closely matches existing bidirectional and autoregressive methods in metrics like cross-shot consistency, shot transition control, and text alignment.

Original post →

ShotStream: Real-Time Multi-Shot Video Generation

More from Multimodal