WorldFlow: generative world model jointly predicting video and 3D trajectories, ADE down 54.2%
Yuhao Lu · hf · 2026-11-03
WorldFlow (RoboScience & NUS) is a generative world model that jointly predicts future video and query-based 3D trajectories.
- Method: parameterizes 3D motion as continuous displacement tokens aligned with the video grid, denoised jointly by a unified Mixture-of-Transformers backbone under a rectified-flow objective — coupling explicit geometric dynamics with visual evolution to unlock physical common sense in pretrained video models.
- Scale & eval: trained on 1.44M object-level samples across simulated and real-world domains; introduces WorldFlowBench (1,678 samples from 11 sources, including spatial reasoning tasks).
- Results: lowest average displacement error (ADE) and final displacement error (FDE) across all three evaluation sets; on SpatialCog it achieves 0.060 m ADE, a 54.2% reduction vs. the strongest external baseline, and cuts ADE/FDE by 55.9%/68.5% vs. its own video foundation model.
More from Multimodal
- Art Nouveau Fairytale prompt template: flowing lines, organic shapes, dual-color palettes — LudovicCreator · 2026-11-03
- Free image-to-3D ComfyUI workflow runs on just 8GB VRAM with a local Gradio app — TheLocalLab · 2026-11-03
- H3 workflow nails 1-4 character swaps with auto-assembled prompts, rivaling closed models — Cukidzsi · 2026-11-03
- Mago open-sources a ComfyUI-based AI video studio for script-to-shorts with consistent characters — devdef · 2026-11-03
- Midjourney --sref style codes shared: minimalist miniature figure photography look — sergeantsref · 2026-11-03
- AI-generated Sun Wukong training short wows Reddit with fur detail and martial arts motion — lucky-plume · 2026-11-03