HSImul3R makes 3D reconstruction simulation-ready, cutting penetration rate from 69.5% to 22.9%
机器之心 · wechat · 2026-09-20
Daxiao Robotics, NTU S-Lab, and Shanghai AI Lab released HSImul3R (accepted to ECCV 2026), targeting the perception-simulation gap in 3D reconstruction: visually correct scenes often collapse or interpenetrate once placed in a physics simulator.
- New criteria: the first simulation-ready human-scene interaction reconstruction framework for uncalibrated sparse views (plus monocular video), adding gravity stability, real contact, and interaction stability as core metrics, with a new HSIBench (19 objects, 300 interaction instances).
- Physics-in-the-loop: scene-targeted RL optimizes human motion to truly interact with the scene, while Direct Simulation Reward Optimization uses simulation feedback to fix scene geometry — the simulator shifts from judge to active supervisor.
- Results: interaction stability of 53.68%/30.56%/13.92% on Easy/Medium/Hard (vs HSfM's 10.52%/4.50%/2.66%); penetration rate cut from 69.51% to 22.90%.
- Sim2Real: optimized motions were retargeted to a Unitree G1, trained into whole-body policies, and deployed on the real robot — suggesting internet human videos could become learnable robot skill assets.
More from Embodied
- MIT builds 2nm polymer nano-spring device that mimics neurons for brain-inspired computing — MacrinePhD · 2026-09-20
- Faraday Future unveils nine AI robots and four industry solutions — emmanuelvivier · 2026-09-20
- Nvidia releases SONIC, a humanoid whole-body controller trained on 100M motion sequences — emmanuelvivier · 2026-09-20
- Ant Group's LingBot-Map, ECCV 2026 Oral: streaming 3D reconstruction from one RGB camera at ~20 FPS — jiqizhixin · 2026-09-20
- World Labs CEO Alex Kendall to Headline Long Horizon Physical AI Summit — alexgkendall · 2026-09-20
- Dev shows robot arm controlled by Claude, quips robotics is 'far too easy' and returns to GPU kernels — KuterDinel · 2026-09-20