Training Humanoid Robots in 3D Scans: Zero Real-World Fine-Tuning
lukas_m_ziegler · x · 2026-08-10
The author shares a practice of training humanoid robots entirely via Reinforcement Learning (RL) within a 3D scan of a real office.
Traditional RL requires extensive trial and error. Doing this in the real world risks hardware damage and consumes reset time, making simulation necessary. However, standard simulations use untextured geometry, meaning the robot only learns depth and structure without grasping real-world materials or lighting, leading to poor generalization.
To bridge this gap, the author used @NianticSpatial's Scaniverse to perform a 360° walkthrough of the deployment site. This generates a metric-scale photorealistic 3D Gaussian Splatting environment and extracts collision meshes, ensuring exact alignment between vision and physics. The setup integrates directly with NVIDIA's simulation stack, achieving a Sim2Real transfer with zero real-world fine-tuning.
More from Embodied
- Rabbit R1 Review: A $199 Voice Remote for Your AI Stack, Shines When Wired to Agents — SimonBalmain · 2026-08-10
- TriWorldBench Launches as First Tri-View Evaluation for Embodied World Models — 量子位 · 2026-08-10
- Physical AI Exposes Three 'Fracture Layers'; DeepRoute Launches First Physical AI Lab in China — 量子位 · 2026-08-10
- Chinese Makers Account for 97% of Global Humanoid Robot Shipments, Report Says — rohanpaul_ai · 2026-08-10
- GitHub Repo Indexes Open-Source Hardware: Robotics, 3D Printing & Biotech — tom_doerr · 2026-08-10
- Barclays: Humanoids Are a Compute Story, Set to Ignite AI Infrastructure Demand — coinfanking · 2026-08-10