OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation
Wenxue Li, Peiyan Guan, Haoyang Jiang, Junxian Cai, Hualuo Liu, Chunjie Zhang, Chong Guan, Songlian Li, Taiyi Wu, Yongjian Yu, Xiaotong Zhao, Alan Zhao, Eric Liu, Xi Chen, Yu Liu, Lei Zhu
cs.CV
2026-09-19
OmniVBench covers 7 families and 18 R2V tasks with 12,172 factor checks. Seedance 2.5 scores 72.68, MiniMax H3 72.41; content is strong, while motion, narrative, and routing lag.
Reference-to-video has moved past "keep this face" into action, camera, style, line art, storyboards, next-shot continuation, and several references at once. Existing benchmarks still center on content, and scores still ask whether the output looks similar overall. Omni R2V fails in a more specific way: a factor that should be kept is dropped, a factor that should be stripped rides along, or the instruction binds to the wrong object.
Training data is equally fragmented. OpenS2V and Phantom-Data are large on paper, but they cover few modalities and tasks, and some do not ship processed reference–video pairs. Tencent's Online Video group and HKUST(GZ) put a benchmark and a 340K-sample set on the same taxonomy.
OmniVBench has seven families and 18 fine tasks. Five single-reference families: content (object, character, scene), motion (action and camera), style, structure (greybox, line art, rough boards), and narrative (multi-panel boards, story, preceding shot). Two multi-reference families: multi-content composition, and cross-aspect pairs such as content with motion or structure. Prompts withhold information that should be read from the reference, so the model cannot pass by reading the text. The suite has 813 cases.
Scoring splits three ways. Reference Fidelity (RF) asks whether designated factors land in the video. Instruction Realization (IR) asks whether factors are disentangled and routed to the right target, and whether the prompt's operations happen. Video Quality (VQ) uses DOVER++, Aesthetic Predictor V2.5, and UnifiedReward 2.0, independent of the reference. RF and IR are driven by 12,172 human-checked checklist items. Gemini scores RF on a 1–5 scale and IR in three bins, then equal-weight aggregation walks up the hierarchy.
The Omni-R2V set has about 339,570 processed samples, mostly from professional footage. Pairs come from cross-clip matching (same identity or style) or inverse construction (extract line art, generate a simplified structure). Gemini captions references and targets; DeepSeek writes the training prompt so each reference's role is explicit.
Seedance 2.5 leads closed models at 72.68. Open-source MiniMax H3 is 72.41, essentially tied. Seedance 2.0 (70.91), Happy Horse 1.0 (70.89), and Gemini Omni (70.76) follow. The next open model, Bernini, is 56.95: the top of the open and closed lists have converged, the mid-pack has not.
No model wins every family. Content scores sit above motion, style, structure, and narrative. IR splits further. Several models comply with the target prompt while scoring much lower on disentanglement and routing, especially on multi-content and cross-aspect cases. Following the text and dropping the wrong part of a reference are different skills.
| Model | Content | Motion | Narrative | Overall |
| Seedance 2.5 | 78.88 | 66.10 | 73.97 | 72.68 |
| MiniMax H3 | 78.86 | 65.32 | 73.90 | 72.41 |
| Kling 3.0 Omni | 75.99 | 64.20 | 68.59 | 68.59 |
| Bernini | 69.27 | 55.92 | 42.02 | 56.95 |
| UniVideo | 66.01 | 44.52 | 36.97 | 49.69 |
On 100 cases and 965 outputs, automatic scores correlate with humans at Pearson 0.81 / 0.78 / 0.86 for RF / IR / VQ. Factor checklists beat holistic judging on Spearman (RF 0.77 vs 0.67, IR 0.74 vs 0.69).
R2V work now has a table that can mark "copied the wrong factor", not only a global similarity score. Training gets processed samples for motion, style, structure, narrative, and multi-reference without forcing every lab to pair raw footage. For products, identity-style content reference is comparatively strong; camera, story, and cross-reference binding are not. A character-consistency demo is not an omni system.
The paper ships a benchmark and a dataset. It does not train a new generator on Omni-R2V that beats Seedance. The numbers diagnose. They do not ship a model.
Gemini still grades RF and IR. Human correlation is high, but the judge has its own bias. Source footage is mostly in-house professional material, which may not match user-uploaded clips. 813 eval cases are small next to 340K training items, so per-task variance will be noisy. Duration, resolution, and reference visibility are uneven across families. There is no reported finetune of an open backbone on Omni-R2V, so the training value of the set is still indirect. Checklists unpack factors; temporal consistency across shots can still be averaged away.