VLA-Replica: a $-low-cost real-world VLA benchmark where NVIDIA GR00T N1.7 matches π₀ with 50 demos

YuXiang_IRVL · x · 2026-09-20

UT Dallas's Intelligent Robotics and Vision Lab released VLA-Replica, a low-cost, easily reproducible real-world benchmark for evaluating vision-language-action (VLA) models, plus a public leaderboard.
Built from off-the-shelf parts (SO-101 follower arm, light box, cameras), a user with no prior experience assembled it within one hour, enabling consistent policy evaluation across labs worldwide.
The benchmark spans 10 diverse manipulation tasks with a small demonstration dataset, covering both in-distribution and out-of-distribution evaluation protocols.
Fine-tuned with only 50 demos, NVIDIA's GR00T N1.7 VLA model performed on par with π₀ in ID settings and close to π₀.₅ in OOD scenes. Paper, code, checkpoints, and setup guide are available; accepted at the RSS 2026 Data-Centric Robotics Workshop.

More from Embodied