VideoGen-Agent: RL-trained tool-using agent lifts video generation by 19.1 points
机器之心 · wechat · 2026-10-09
VideoGen-Agent, built on Qwen3-VL-8B-Instruct, learns via SFT plus multi-task RL (rewards weighted 0.1/0.4/0.5 for call validity, tool use, and video quality) to orchestrate retrieval, simulation, and verification tools before generating video. On the new 600-prompt VABench it scores 75.6 with Toolset1 vs a 56.5 base generator (+19.1), rising to 86.1 after swapping in stronger generation tools without retraining the agent. In human evals, 84.3% preferred its output over Seedance 2.0. Ablations show RL adds 6.4 points over SFT, and the policy zero-shots to 73.2 on unseen procedural-knowledge-plus-identity combos, spontaneously combining text and image retrieval in 84% of traces.
More from Multimodal
- MiniMax-H3 hailed as favorite open release of the year, runs fully local — AdinaYakup · 2026-10-23
- Michal Malewicz experiments with depth maps in his creative workflow — michalmalewicz · 2026-10-23
- One-Shot Tunable Style Transfer on Flux3 Via Reference Latent + Node — Winter_unmuted · 2026-10-23
- PhysVidBench accepted at EMNLP 2026: do video gen models grasp physical commonsense? — aykuterdemml · 2026-10-23
- EMNLP 2026 paper teaches multimodal LLMs to get cartoons via incongruity-resolution supervision — aykuterdemml · 2026-10-23
- A gorgeous Midjourney style: sref code 820221253 for dreamy lavender watercolor looks — umesh_ai · 2026-10-23