BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender
Yolo Y. Tang, Daiki Shimada, Jiayue Meng, Jing Bi, Pinxin Liu, Yicheng Wang, Yunzhong Xiao, Zhangyun Tan, Zeliang Zhang, Chao Huang, Susan Liang, Qianxiang Shen, Luchuan Song, Ali Vosoughi, Mingqian Feng, Melika Filvantorkaman, Chenliang Xu
cs.CV
2026-09-14
BVB asks agents to rebuild indoor videos in Blender under a $3 cap; GPT-6-Astra-high leads at 70.07 overall, with 88.6 visual similarity but only 53.7% fact retention.
Video understanding is still mostly question answering. A correct option can come from answer priors or a single frame. That does not prove the model tracked how a scene changes over time. If an agent understood the video, it should be able to rebuild it in code: where objects sit, how the camera moves, and in what order events happen. Those details are hard to fake with a multiple-choice prior.
Multimodal agents can already write Blender code. Existing demos often use chosen scenes, repeated human guidance, and external asset libraries. They do not show reliability on new rooms, how families differ, or how close the reconstruction is in layout and motion.
BVB (Blender-VideoBench) is built on VSI-Bench’s egocentric indoor clips from ARKitScenes, ScanNet, and ScanNet++: 288 test scenes and 5,130 spatiotemporal questions. Every agent uses the same Docker sandbox and Mini-BVB harness with two actions, fetch frames by timestamp and run Python/Blender, in a no-network container with 2 CPUs and 4 GB RAM. External assets are banned. Geometry must come from primitives, the camera must be animated along the source path, and the deliverable is a renderable .blend. The only budget is $3 per scene.
Scoring has two axes. Dual VQA (DV) asks a VLM judge the same questions on the source and the reconstruction, and counts retention only on items the judge got right on the source. Latent Similarity (LS) compares 64 source frames with 64 EEVEE renders using frozen V-JEPA 2.1 ViT-G. Overall is the square-root mean of the two axes, which penalizes a system that is strong on only one.
Fifty-one configurations from ten families span 47.50–70.07 Overall, 38.6–53.7 DV, and 57.2–88.6 LS. GPT-6-Astra-high leads at 70.07 Overall, 53.7 DV, and 88.6 LS, then GPT-5.6-Sol-xhigh at 67.49 and Grok-4.6-xhigh at 67.17. The best open-weight run is GLM-5.3-Flash-xhigh at rank 12 and 63.96. Of 1,827 source-correct questions, the top model still answers 981 after reconstruction and loses 846.
| Config | Overall | DV | LS | $/scene |
| GPT-6-Astra-high | 70.07 | 53.7 | 88.6 | $1.258 |
| GPT-5.6-Sol-xhigh | 67.49 | 51.7 | 85.4 | $0.778 |
| GLM-5.3-Flash-xhigh | 63.96 | 50.8 | 78.7 | $0.024 |
Retention is uneven. Object size keeps 63.0% on average and route planning 58.5%; appearance order keeps 16.1% and object count 29.3%. Extra provider reasoning lifts LS more reliably than DV, which can stay flat or fall. Longer runtime is not a reliable Overall predictor. Fifteen blind raters recover the Overall order of five configs exactly (Spearman ρ=1.00). At scene level they track LS (ρ=0.83) far more than DV (ρ=0.11).
Programmatic reconstruction turns “understand the video” into an editable 3D scene, which is harder to game than a quiz. For video-agent work, the current frontier can already assemble rooms that look right, while whole-scene facts such as counts and appearance order remain the main hole. More thinking polishes geometry and materials; it does not automatically write a more honest scene ledger. Cost is usable: Sol-xhigh reaches 96% of the top Overall at 62% of Astra’s spend, and Flash-xhigh reaches 91% at $0.024 per scene.
All scenes are indoor and egocentric. Primitive geometry systematically drops real materials and complex shapes, so high LS can partly mean “both are low-poly rooms.” DV inherits the VLM judge; items the judge misses on the source never enter retention. Humans side with LS, so Overall must be read as a blend of look and fact, not as a single ability. The $3 cap and asset ban keep the comparison fair and also cap the ceiling at rooms built from blocks. Sony funded the work; the roster skews toward flagship proprietary models.