StreamEval from Tsinghua, NVIDIA, CMU and Berkeley exposes gaps in streaming video generation across 38 models
jiqizhixin · x · 2026-11-10
Researchers from Tsinghua, NVIDIA, CMU and Berkeley released StreamEval, a benchmark for interactive streaming video generation: does a model actually react when instructions change, or just keep producing convincing frames? Polished finished clips can hide missed instructions, delayed reactions and inconsistent state — appearance-only evaluation misses these failures. The benchmark schedules online inputs so models only see information released so far: 300 scripted cases across 5 tasks, videos of 15–300s, 1,347 planned input events, 10 metrics over 4 axes, 3 repeat runs. Testing 38 models across 49 model-task configurations reveals substantial gaps between looking plausible and responding timely and correctly.
More from Multimodal
- Is ControlNet Obsolete? New DiT Image Models Ship Without It — Manicarus · 2026-11-10
- Full prompt revealed: how to recreate the Van Gogh convenience store in MiniMax H3 Image — mhdfaran · 2026-11-10
- 10 open-source GitHub repos for voice AI: from local ASR to zero-shot voice cloning — Shruti_0810 · 2026-11-10
- Fable 5.2 generates animated nature scene with ~70k trees from one prompt — Merzmensch · 2026-11-10
- MOSS-Image roundup: Control LoRAs, 461 new characters, and 20-minute LoRA training on a 5090 — optimisticalish · 2026-11-10
- Reddit hot take: World Labs' deterministic camera control beats rolling the dice with AI video — Spiritual_Skirt_9312 · 2026-11-10