CUHK's VideoCoCo: executable code as CoT lifts VBench-2.0 average by 25.7 points
机器之心 · wechat · 2026-08-16
Researchers from CUHK and USTC propose VideoCoCo, targeting the "causal opacity" problem in text-to-video: a one-line prompt says what happens but omits speeds, ordering, and interactions. The system uses a dual-agent design: a Coding agent first writes a self-contained Blender program from the prompt and runs it in a sandbox, producing a deterministic white-model video that fixes camera, motion, and event rhythm; a Visual agent then re-renders the rough simulation into photorealistic footage guided by the white model and edit instructions. The code acts as an executable, checkable "process-level chain of thought."
Results: on PhyGenBench (GPT-4o as judge), it lifts the baseline OmniWeaving from 0.475 to 0.558, beating the open-source Wan2.2-TI2V-5B at 0.544, ranking first in mechanics, optics, and material; on VBench-2.0 the average rises from 52.18% to 77.88% (+25.7 points), with mechanics at 92.31%. Ablations show that simply conditioning on the executable draft without fine-tuning the editor already improves the score from 0.475 to 0.506, indicating most of the physics-consistency gain comes from the draft itself rather than extra training.
The work extends the team's ECCV 2026 CoCo approach from static previews to continuous dynamics. The authors argue executable code could become an intermediate language for video generation — denser than text plans, more complete than keyframes, and auditable/correctable — as a concrete form of neuro-symbolic integration. Limitations include the physics Blender can express, script stability, and white-model-to-video alignment. Paper and code are open-sourced.
More from Multimodal
- USTC paper uses speculative sampling to cut latency in interactive video generation — ustc · 2026-10-16
- 800,000 Gaussian splats run in WebGPU; Casado sees near-infinite scaling — generativist · 2026-10-16
- Nano Banana 2.1 image generation now live in Perplexity Computer for all users — ChrisUniverse · 2026-10-16
- Demo shows precise camera-path control in 3D scenes from a single input image — XRarchitect · 2026-10-16
- Train Your Own LTX IC-LoRA With Grok Doing the Heavy Lifting — PurzBeats · 2026-10-16
- Hugging Face launches Open TTS Leaderboard: objective metrics to rank 8K+ open TTS models — dl_weekly · 2026-10-16