StreamEval benchmarks streaming video generation: best model scores just 36.90 on task execution
StreamEval Team · hf · 2026-10-22
StreamEval, from Tsinghua, NVIDIA, CMU, UC Berkeley, Princeton and NUS, is a unified benchmark for interactive streaming video generation spanning five tasks (avatar dialogue, avatar performance, world generation, instruction editing, reference editing), with 300 scripted test cases (15-300s) scored on task execution, state consistency, visual fidelity, and audiovisual coordination. Evaluating 37 models shows high visual quality doesn't guarantee reliable request execution or sustained state consistency — the highest task-specific score is just 36.90. Leaderboard and data are open-sourced.
More from Multimodal
- fal employee shows Claude + fal generating a game in 60 minutes — OdinLovis · 2026-10-22
- Indie dev uses AI to 1:1 recreate his own RV for his fourth game — ezshine · 2026-10-22
- User Shows Off Cinematic AI Video Made with Seedance 2.5 — SimplyAnnisa · 2026-10-22
- Fudan's TextRIRO enables scene text editing with self-paired real data only — Fudan-University · 2026-10-22
- Spline launches ChatGPT integration to create and edit interactive 3D scenes in chat — algo_diver · 2026-10-22
- Ant Group's Ming-3.0-TTS Tops Pronunciation Robustness Leaderboard at 93.2%, MIT-Licensed — lmoroney · 2026-10-22