FilmBench evaluates cinematic video generation with film-school shot lists and 35 metrics
Shengyi Wang · hf · 2026-07-28
The authors introduce FilmBench, a film-grade benchmark for cinematic video generation.
- It covers both text-to-video (T2V) and reference-to-video (R2V).
- Prompts are reverse-engineered from award-winning films across 20 cinematic genres and aligned with real shot lists.
- The benchmark is co-developed with directors and faculty from the Beijing Film Academy and the Hujing Digital Media & Entertainment Group studio.
- Evaluation uses a three-level cinematic taxonomy with 3 axes, 12 components, and 35 T2V + 3 R2V-only sub-metrics.
- The team also builds an in-house expert-grade evaluation agent and open-sources FilmOps, its core operator suite.
- On 9 T2V and 7 R2V models, the evaluator matches human ranking very closely, with Spearman correlation of 0.95 and 0.96.
- Scores are lower than on prior web-style benchmarks, revealing gaps in dynamic aesthetics and a strong drop from single-shot to multi-shot generation.
More from Multimodal
- Topview launches Film Studio with 3D blocking and micro-expression controls — XFreeze · 2026-07-28
- GaussianGPT uses autoregressive next-token prediction to generate 3D Gaussian scenes — rsasaki0109 · 2026-07-28
- Testing Style LoRAs: How to Isolate Style from Content in Image Generation — Dark_Sytze · 2026-07-28
- VLM evidence attribution improves with quote-and-retrieve, not bounding boxes — Zhuchenyang Liu · 2026-07-28
- Rethinking CFG in OPD: branch-aware distillation fixes Negative Branch Asymmetry — Bingnan Li · 2026-07-28
- Oxygen-TryOn is a fashion-native model for multi-item virtual try-on — JD-company · 2026-07-28