TempCloze: Can Video-LLMs Identify the Missing Middle?
Wenqi Pei, Henry Hengyuan Zhao, Yilai Liu, Jiahao Meng, Han Chen, Ziyu Wang, Hongyang Du
EMNLP 2026 Findings
cs.CV, cs.AI
2026-09-02
TempCloze asks Video-LLMs to pick the true missing middle from four same-video clips. Across 31 models, temporal alignment is the bottleneck.
Most temporal tests for Video-LLMs still go through language: pick a text option, or write a caption. Wording, answer correlations, and language priors can carry a score. Prior work has shown that a text-only model can beat random by more than 25% with no video at all.
TempCloze turns the question into a visual cloze. The model sees the beginning and ending of a clip and must pick the true missing middle from four candidates. The right choice is not merely related to the video. It has to sit in the correct temporal slot between the two visible ends. Every distractor comes from the same source, so scenes and objects are shared and appearance matching is a weak cheat.
The benchmark keeps 1,521 videos from seven public sources, mostly long-take and egocentric footage: LVD-2M 515, EgoLife 437, MiraData 198, FAVOR 145, plus CaReBench, Video-TT, and Daily-Omni. Median duration is 30.0 s, mean 31.3 s. Filters drop clips outside 12–90 s, use GPT-o3 on dense captions to reject poor cloze cases, then cut low bitrate, low sharpness, and near-static clips via Farnebäck optical flow. The missing gap is drawn from the central 50% of the timeline and covers 20%–40% of the video, leaving context on both sides.
Each video is instantiated three times, always with same-source distractors:
The default sample is 16 frames per clip, 96 frames across the six clips in one dimension. A boundary-frame check removes the trivial pixel-match shortcut. A human baseline on 100 random videos scores 96 / 98 / 97 on the three dimensions, mean 97%.
Ten proprietary and 21 open-source models. Chance is 25%.
| Model | Semantic | Alignment | Progression | Mean |
| Human (100 videos) | 96.00 | 98.00 | 97.00 | 97.00 |
| Seed1.8 (think) | 95.07 | 76.92 | 93.75 | 88.58 |
| Qwen3.5-Plus (think) | 90.72 | 76.32 | 90.20 | 85.74 |
| Gemini 2.5 Pro (think) | 90.33 | 66.67 | 67.78 | 74.92 |
| GPT-5.4 | 68.11 | 37.41 | 65.15 | 56.89 |
| Qwen3.5-397B-A17B | 75.94 | 50.62 | 78.24 | 68.27 |
| Proprietary avg. | 70.73 | 48.13 | 67.72 | 62.19 |
| Open-source avg. | 34.00 | 26.54 | 36.97 | 32.51 |
| Grok 4.1 | 24.52 | 24.06 | 23.73 | 24.11 |
Alignment is the bottleneck. Seed1.8 still sits at 76.92 Alignment versus 98 for humans. The best open model on that axis, Qwen3.5-35B, reaches 51.55. Complete 3/3 accuracy averages 33.24% proprietary and 7.15% open-source, against 92% human. Turning think off on Seed1.8 drops Alignment from 76.92 to 61.93. Grok 4.1 hugs the 25% chance line on every dimension.
Errors are structured. Expanded dominates Alignment failures: Seed1.8-I picks it in 75% of those errors. Reversed dominates Progression failures: 67% of GPT-5.4's process errors. On the mixed-dimension subset, Alignment distractors collect the largest error share.
Choices are brittle. Permuting option order on the Hard subset moves accuracy by less than 4 points, but the selected clip changes often (CFR 32.4%–60.7%). Models lean on the beginning more than the ending: Seed1.6 Progression is about 80% with beginning only and 46% with ending only. Stretching the visible span or raising sampling from 8 to 20 frames per clip hurts Alignment, as boundary evidence is diluted. Extra test-time samples lift absolute scores and leave the dimension ranking unchanged.
This is a temporal ruler that is harder to game with language. A single VideoQA number can hide the gap between recognizing an event and placing it. Alignment work needs a sharper boundary representation and two-sided context, not only forward continuation from the opening clip.
If the product is video localization, long-take editing, or an egocentric assistant, do not assume a stronger Video-LLM already "understands time." Most open 7B–8B models still sit near chance on this test.
TempCloze is a diagnostic, not a full video-understanding suite. It does not cover open-ended generation, dialogue, audio, or free future prediction. Scores are relative to these distractors. Same-source candidates make the task stricter: an Alignment foil can look like the right event with the wrong span, by design. Sources skew long-take and egocentric. APIs and decoding change, so the table is a snapshot. The human baseline uses 100 videos.