The Smoothest Robot Rollout of 127 Episodes Still Failed: Motion Smoothness Metrics Measure Motion, Not Success
Scobleizer · x · 2026-09-13
Harpreet Sahota, Hacker-in-Residence at Voxel51, scored all 127 episodes of a cube-in-bowl task from RoboLab-EgoX on motion smoothness (SPARC, LDLJ, etc.) and found the 8th-smoothest rollout in the batch actually failed the task — zero flags, no anomalies.
Key takeaways:
- Smoothness metrics measure how the arm moved, not whether the cube landed in the bowl. A fluid path to the wrong place scores as flawless.
- Score for triage, not gates: smoothness tells you which episodes to review, never which ones passed.
- The article includes a reproducible workflow: picking a single task without downloading the full 21.8GB dataset, building the dataset and motion signals, scoring, and working around a 5.4-second episode.
More from Embodied
- Walden Robotics CTO: general-purpose robot manipulation is having its 'GPT moment' — adnothing · 2026-10-30
- Gym-Anything Team Stopped at 25 Simulations Due to Resource Limits, Seeks Collaborators — wellecks · 2026-10-30
- Gym-Anything Adds Audit Agents That Try to Break Simulations With Legal Robot Actions — wellecks · 2026-10-30
- NVIDIA researcher: train robot policies with privileged sim info, improving Pi 0.5 — wellecks · 2026-10-30
- Leveraging Simulator Privileged Information: Three Methods to Improve Robot Models — wellecks · 2026-10-30
- CMU Auto-Builds 25 Robot Simulations with Agents; Frontier Model Solves 0 of 25 Test Tasks — wellecks · 2026-10-30