NVIDIA's Foundation4D turns multi-camera video into 4D perception — no LiDAR needed
jonstephens85 · x · 2026-10-29
NVIDIA Research released Foundation4D, a unified 4D perception and reconstruction foundation model for autonomous driving that works without LiDAR input.
- What it does: a shared backbone fuses multi-camera video over time to output geometry, 3D detection & tracking, semantic occupancy, and Gaussian Splatting reconstruction; video-level processing refines camera trajectories and links objects into tracks.
- Results: depth estimation hits AbsRel 0.09 / δ1.25 91.97% on Waymo & nuScenes, beating DVGT, LiAuto-GeoX and others; camera trajectory ATE drops to 23.6cm vs DVGT-1's 154.5cm under Waymo PGO+LM.
- Releases: the PAI-Foundation4D benchmark with dense depth and occupancy annotations for 1,253 driving clips, plus white paper and dataset.
- Use case: predicted geometry and trajectories serve as priors for LiDAR-free NuRec reconstruction, simplifying dataset preparation.
More from Embodied
- Robotics researcher gifts world-model-themed chocolate with autoregressive bite-prediction diagram — breadli428 · 2026-10-29
- Mundane pitches a 'second body' robot that takes the shock or blast instead of tradespeople — Scobleizer · 2026-10-29
- Empatica launches WellPD, an FDA-cleared wearable for at-home Parkinson's monitoring — RosalindPicard · 2026-10-29
- QUAN paper: real-valued quaternion convolution nets SOTA in oriented detection — Bryce Grant · 2026-10-29
- Neuralink says brain implants now turn neural signals into speech at conversational speed — Polymarket · 2026-10-29
- Robotics team hosts Sim2Real playoff: win in sim, control a real robot — cixliv · 2026-10-29