NVIDIA OmniDreams: real-time generative world model for closed-loop AV simulation, at 16 GB300 per video
yassineyousfi_ · x · 2026-10-20
- NVIDIA published a technical report on OmniDreams, a foundation generative world model mid/post-trained from the Cosmos diffusion model on 21k hours of driving data, generating action-conditioned photorealistic video in real time.
- It autoregressively conditions on past frames, simulator state, and driving actions, synthesizing long-tail phenomena (extreme weather, unpredictable agents) that reconstruction-based neural simulators can't, for closed-loop evaluation with the Alpamayo 1 policy model.
- Discussants note the 32B version is arguably the best world model for action conditioning and coherent rollout — but generation reportedly requires 16 GB300 GPUs for one video; the hard part is accurate action conditioning, not pretty frames.
More from Embodied
- New "Embodied Olympics" Benchmark: GPT and Claude Agents Play Soccer and Ping Pong — ManlingLi_ · 2026-10-20
- SF founders go deep in Shenzhen's Huaqiangbei building BCIs and wearables — LoganGrasby · 2026-10-20
- MicroAGI trains a handyman model with tens of thousands of hours of video — isnit0 · 2026-10-20
- Uber-linked group secretly campaigns to ban robotaxis in Washington D.C. — gabriberton · 2026-10-20
- Meta admits smart glasses were recording conversations without the wake word — Resident-Fall4351 · 2026-10-20
- Wayve demos its Fiat 500e autonomous prototype on Turin's iconic Lingotto rooftop track — alexgkendall · 2026-10-20