LeRobot Unifies Robot Evaluation CLI
RemiCadene · x · 2026-07-15
LeRobot 0.6.0 introduces a unified lerobot-eval CLI to evaluate various VLA robot benchmarks, eliminating the hassle of juggling multiple repositories and environments.
This release adds 6 simulation benchmarks, each equipped with dedicated documentation, Docker images, and CI-verified SmolVLA baselines:
- LIBERO-plus: 10,000 perturbation variants across 7 dimensions to observe policy failure conditions
- RoboTwin 2.0: 50 bimanual tasks based on SAPIEN, with 100,000+ trainable trajectories on the Hub
- RoboCasa365: 365 kitchen tasks across 2,500 procedurally generated kitchens
- RoboCerebra: Long-horizon tasks chaining 3–6 sub-goals
- RoboMME: Memory tests including counting, tracking, and imitation
- VLABench: Knowledge and reasoning tasks, from physics to end-to-end "making coffee"
Combined with LIBERO, Meta-World, and NVIDIA IsaacLab-Arena, there are now 9 benchmark families runnable under a single toolset.
More from Embodied
- Claude Code finds root backdoor in Honda Civic head unit, then debloats it — IndraVahan · 2026-10-26
- Temple health wearable opens pre-orders, users ask: why wear it before benefits are proven — abhishekcode42 · 2026-10-26
- FLASH: GPU-native deformable sim learns cloth folding in minutes on one RTX 5090 — ChongZzZhang · 2026-10-26
- Luke Wroblewski spots it: iPhone Duo's corner radius mirrors the original iPhone — LukeW · 2026-10-26
- Humanoids shattered forecasts: half-marathon record in 2026, 100m faster than Bolt — ben_j_todd · 2026-10-26
- Humanoid walking is continuous correction — watch recovery, not locomotion, in demos — Olivier__OG · 2026-10-26