Unreal Engine pipeline yields 8.7K+ hours of action-conditioned video for world-model pretraining
udmrzn · x · 2026-09-06
- An arXiv paper presents an Unreal Engine-based pipeline generating 8.7K+ hours of synchronized multi-view video with frame-aligned actions and camera states for pretraining action-conditioned world models, solving the lack of control-signal supervision in real-world video.
- The pipeline runs in two stages: Stage I executes real-time physics in PIE, recording per-frame character states, control inputs, and camera states; Stage II replays trajectories in a fresh engine process for high-quality offline rendering via Movie Render Queue.
- A distributed system handles cache-aware task partitioning, node-local slot scheduling, automated scene screening, aesthetic/luminance filtering, partial-output recovery, async upload, and cluster health monitoring. The cluster spans 25 servers with 8 NVIDIA RTX 5090s each, processing 2,384 scene tasks.
More from Infra
- llama.cpp team launches Llama: one-click local AI on your Mac menu bar — Hugging Face · 2026-10-23
- China's daily token use hits 500T, up 4x as average price doubles to ¥13.46 — teortaxesTex · 2026-10-23
- New all-reduce kernels deliver 12% decode speedup for DeepSeek-V4.1 on PCIe-connected GPUs — AlpinDale · 2026-10-23
- Offshore oil rigs could be converted into AI data centers powered by stranded gas — clemnt · 2026-10-23
- Same native UI, two stacks: Swift build is ~80MB Mac-only, Zig build is ~10MB cross-platform — jedisct1 · 2026-10-23
- SurfSense: an open-source local NotebookLM with MCP agents and a prompt-caching lesson — FurtiveMirth · 2026-10-23