LeRobot Unifies Robot Evaluation CLI
RemiCadene · x · 2026-07-15
LeRobot 0.6.0 introduces a unified lerobot-eval CLI to evaluate various VLA robot benchmarks, eliminating the hassle of juggling multiple repositories and environments.
This release adds 6 simulation benchmarks, each equipped with dedicated documentation, Docker images, and CI-verified SmolVLA baselines:
- LIBERO-plus: 10,000 perturbation variants across 7 dimensions to observe policy failure conditions
- RoboTwin 2.0: 50 bimanual tasks based on SAPIEN, with 100,000+ trainable trajectories on the Hub
- RoboCasa365: 365 kitchen tasks across 2,500 procedurally generated kitchens
- RoboCerebra: Long-horizon tasks chaining 3–6 sub-goals
- RoboMME: Memory tests including counting, tracking, and imitation
- VLABench: Knowledge and reasoning tasks, from physics to end-to-end "making coffee"
Combined with LIBERO, Meta-World, and NVIDIA IsaacLab-Arena, there are now 9 benchmark families runnable under a single toolset.
More from Embodied
- Ant's Afu health AI hits 150M users, unveils AI+hardware health alliance at Bund Summit — APPSO · 2026-09-11
- Johns Hopkins Launches Full-Stack Hands-on Robot Learning Class with SO-101 Arm Kits — _krishna_murthy · 2026-09-11
- SyncWorld: In-Context Robot World Model Simulates Unseen Views and Embodiments Zero-Shot — ChongZzZhang · 2026-09-11
- A 3D Pose Dataset for Dogs Released — ducha_aiki · 2026-09-11
- Swaayatt demos autonomous driving at 52 km/h on mountain roads, self-recovers after skid — sanjeevs_iitr · 2026-09-11
- AUAR's MicroFactory brings a deployable robotic wood-panel factory to the construction site — lukas_m_ziegler · 2026-09-11