Ropedia Xperience 2 releases 1,589 3D scans and 4,230 egocentric recordings for physical AI scaling
liuziwei7 · x · 2026-11-16
NTU S-Lab's Ziwei Liu team released Ropedia Xperience 2, a 'gold standard' dataset for scaling physical AI environments and behaviors:
- Scale: 1,589 released metric-scale 3D scans (596 locations) and 4,230 egocentric recordings covering 74 tasks (131 hours), captured with a handheld LiDAR scanner plus head-mounted multi-camera rig — no physical robots or motion-capture infrastructure needed.
- Pipeline: reconstructs meshes and 3D Gaussian Splatting models, estimates camera and hand/body motion, generates hierarchical language annotations, and registers recordings to scene geometry.
- Reusability: each capture doubles as robot-learning data — scenes become interactive simulation environments, human motion is retargeted to whole-body trajectories (sub-centimeter 0.40–0.55 cm fingertip tracking on a simulated humanoid), and virtual teleoperation plus automated long-horizon trajectory generation (8-subtask, 30/30 success) yields extra behaviors.
- Benchmarks & findings: depth/geometry, 3D detection and spatial intelligence benchmarks reveal counterintuitive results — denser input views don't consistently help 3D geometry estimation (CUT3R's pose AUC drops from 0.4103 to 0.0625; Dust3R/Mast3R hit OOM); all evaluated 3D detection models (incl. LLaVA-3D, SceneScript) fail at tight 3D boxes in cluttered real scenes, none exceeding 7 AP50 and some scoring zero; foundation models handle relative position/height (>74%) but fail at metric distances (<34%) and translations (<55%), lacking true metric grounding.
The paper proposes joint capture of environments and behaviors as a gold standard, laying the data foundation for scaling laws over environments and behaviors.
Related event: NTU S-Lab and Ropedia Release Ropedia Xperience 2 Physical AI Dataset(2 posts)→
More from Embodied
- Impulse-bought robot arm at new Austin Micro Center to automate a 3D print farm overnight — Sentdex · 2026-11-16
- Joe Rogan praises Tesla FSD one-button drive home, urges Tesla to license it — elonmusk · 2026-11-16
- Figure.AI humanoid robot filmed 'leaving the company' in viral clip — Distinct-Question-16 · 2026-11-16
- A beginner's guide to training VLAs: microVLA dev shares low-compute training intuitions — joemeno · 2026-11-16
- Most VLAs lose their base VLM's vision-language skills during robot pre-training, eval shows — DJiafei · 2026-11-16
- NVIDIA AI Day Seoul Draws 2,000+ Attendees Focused on Physical AI and Agents — NVIDIA Blog · 2026-11-16