StreamEval: First open-source benchmark for streaming video generation tests 38 models
kimmonismus · x · 2026-11-06
Researchers from Tsinghua, NVIDIA, CMU, UC Berkeley, Princeton and NUS released StreamEval, billed as the first interactive streaming video generation benchmark.
- Covers 38 models across 300 scripted test cases, with sessions from 15 seconds to 5 minutes
- Five tasks in three categories: Avatar (dialogue, performance, character consistency), world generation (coherent scene transitions), and real-time video editing (preserving unaffected content while editing)
- Ten metrics assess instruction following, visual quality, and scene/state consistency — including keeping an earlier edit intact when the next instruction arrives
- Fully open source: dataset, protocols, code and scoring rules are public
- Key finding: high visual quality and identity retention do not guarantee reliable request execution or sustained state consistency
The leaderboard updates monthly at streameval.ai.
Related event: StreamEval: First Benchmark for Streaming Video Generation(2 posts)→
More from Multimodal
- World Labs launches Frameboy: real-time exploration of its Atlas world model at 15 fps — qtnx_ · 2026-11-06
- Adobe's HueRistic Auto-Tunes Live Video Lighting, Wins UIST Honorable Mention — rufusd · 2026-11-06
- Kling AI team and partners unveil JointLight, a SIGGRAPH Asia 2026 video relighting method — jiqizhixin · 2026-11-06
- AI-generated retro sitcom The Joan & Betty Show releases episode 5: "Peking Duck" — CigaretteBurnXX · 2026-11-06
- AI-crafted fictional game The Last Sticker Shop hides a sticker puzzle — tisch_eins · 2026-11-06
- Gemini Omni Pro spotted internally with 4K support; Omni Flash 1.2 already deployed — koltregaskes · 2026-11-06