Video-Coder trains a coding agent for video generation, gaining 24+ points on PhyGenBench
HuaxiuYaoML · x · 2026-11-10
Paper arXiv:2611.09894 introduces Video-Coder, the first code-driven video generation agent: the model writes and runs code to build a dynamic 3D scene (draft video), which a video generator renders into the final video, with precise control over object count, layout, trajectories, and camera motion.
- Datasets: Video-Coder-SFT-6K and Video-Coder-RL-2K, with a dedicated pipeline
- Training: Qwen3.5-9B, SFT then harness-native agentic RL with dual execution/video rewards and an annealing strategy that phases out execution reward
- Results: +24 points on PhyGenBench, +6 on VBench-2.0
- Full training pipeline released to fill the open-source gap in post-training recipes for small models
Related event: Video-Coder: Agentic Coding for Controllable Video Generation(2 posts)→
More from Multimodal
- Generating game music with Fable 5.2 in just 3 prompts — petergyang · 2026-11-10
- Open-H3-Regenerate-2K: open 3-step video refinement up to 4K, 5x faster than MiniMax API — songhan_mit · 2026-11-10
- Open-source Gemini Flash Image MCP Server adds text-to-image and editing to agents — modelcontextprotocol · 2026-11-10
- Agent draws an oriole's nest stroke by stroke: Fable 5.2's first hand-drawn piece uses 30,000 marks — RileyRalmuto · 2026-11-10
- One-shot WW2 shooter with destructible environments wows users as Spawn adopts Fable 5.2 — TAbrodi · 2026-11-10
- LottieFiles launches Lottie AI: text prompts generate real Lottie files — CodeByPoonam · 2026-11-10