NVIDIA's BirdDreams jointly generates perspective and BEV video for driving world models
nvidia · hf · 2026-11-13
NVIDIA and KAIST introduce BirdDreams, an autoregressive driving world model that jointly generates perspective video and a schematic bird's-eye-view (BEV) video, making scene layout part of the predicted future.
Key ideas
- Learning vehicle motion from perspective video alone is hard: motion is entangled with appearance and vehicles get occluded; prior methods need other vehicles' future trajectories as inputs.
- The BEV renders roads and vehicles as simple shapes at fixed metric scale (fewer occlusions); perspective video retains details like traffic lights. Two video diffusion transformers inform each other via joint attention.
- The predicted BEV provides a geometric interface for ego control: translating/rotating its latest frame along an input trajectory acts as a soft condition for plausible ego motion.
Results: at inference only a single camera image is needed — no HD map or vehicle-state inputs. On PhysicalAI-AV and nuScenes it follows ego trajectories more faithfully than prior driving world models, and human evaluation favors it over a perspective-only baseline for surrounding-vehicle realism.
More from Embodied
- DeepMind unveils Gemini Robotics 2.5: robots tie bows autonomously, 96% pick-and-pack success at Mercedes-Benz — AI_Andrew · 2026-11-13
- CoreWeave Q3 revenue hits $3.66B, up 168%; signs multi-year Wayve training deal — thursdai_pod · 2026-11-13
- Galaxea Co-founder Hang Zhao's CoRL Keynote: Split Robot Intelligence into System 2/1/0 by Timescale — YuXiang_IRVL · 2026-11-13
- Apparent OpenAI hardware unboxing photo leaks: robot arm plus handheld console — yungcontent · 2026-11-13
- FINGR dexterous hand solves a 2x2 Rubik's Cube with continuous finger tricks, wins Best Paper at CoRL 2026 workshop — Scobleizer · 2026-11-13
- Fei-Fei Li amplifies new Atlas demo: 'It is very hard to explain how cool Atlas is' — drfeifei · 2026-11-13