Princeton's VideoGen-Agent Adds 19 Points via Agentic RL Tool Use for Video
princetonu · hf · 2026-09-22
Video generative models still struggle with prompts requiring specialized knowledge, specific identities, physical consistency, or ordered events. Princeton's VideoGen-Agent is a multimodal agent trained via multitask agentic RL to orchestrate augmentation, generation, and verification tools through multi-turn interactions. Training combines SFT on teacher trajectories with RL using a category-aware hybrid reward covering tool-call validity, task-appropriate tool use, and video quality.
They also introduce VABench, a 600-prompt held-out benchmark spanning procedural knowledge, single/multi-entity identity preservation, physical consistency, scene composition, and multi-shot temporal structure. Results: +19.1 points over the base text-to-video model (56.5→75.6); upgrading generation tools without retraining reaches 86.1; human raters prefer the upgraded configuration in 84.3% of comparisons — showing the trained tool-use policy benefits for free from better generation models.
More from Multimodal
- Ditto launches swap-anything video tool to replace any video's main character — yoobinray · 2026-11-12
- Fable 5.2 impresses with pixel art gaussian splats, posts go viral — Flomerboy · 2026-11-12
- Seedream 5.5 nails a consistent character sticker set at just $0.02 per image — azed_ai · 2026-11-12
- Ant Group's open-source Ming-3.0-tts-0.5B tops Open TTS Leaderboard with 0.88 WER — NielsRogge · 2026-11-12
- Viggle Remake recreated in ComfyUI: swap any character in your videos with MiniMax H3 — TheLocalLab · 2026-11-12
- Pika Integrates Seedream 5.5: Split Images Into 100+ Layers for Precise Multi-Turn Edits — aziz4ai · 2026-11-12