CodeCap-Bench: executable code beats natural language as video captions
Jiafu Tang · hf · 2026-10-28
Researchers from NJU, Kling AI, and collaborators propose the Code as Video Caption paradigm and CodeCap-Bench (323 videos): representing video content as executable programs built from graphical primitives, layout rules, and functions of time instead of natural language.
- On downstream QA and video reconstruction tasks, code captions achieve superior reasoning accuracy and reconstruction fidelity by expressing fine-grained spatial structure and temporal dynamics precisely
- Bottleneck: current MLLMs still struggle to generate faithful code captions for complex temporal dynamics
An intriguing new direction for video understanding and captioning evaluation.
More from Multimodal
- Seedance 2.5 hailed as taking AI filmmaking to another level — SimplyAnnisa · 2026-10-28
- Hands-on with Baidu's Unlimited OCR 2: whole docs in one pass, no page limit — Baidu_Inc · 2026-10-28
- Agentic video editing goes pro: senior editors 5x output, costs to collapse — dotey · 2026-10-28
- Alibaba's OvisOCR3: a 0.8B model tops OmniDocBench at 96.74 via multi-teacher distillation + RL — ATH-MaaS · 2026-10-28
- Moving Atoms Engine turns real-world spaces and cities into high-resolution 4D models for sim — Scobleizer · 2026-10-28
- QuiverAI Launches Arrow 2 Vector Model on FLORA with Layered SVG Editing — andpoul · 2026-10-28