VLA-Replica: a $-low-cost real-world VLA benchmark where NVIDIA GR00T N1.7 matches π₀ with 50 demos
YuXiang_IRVL · x · 2026-09-20
- UT Dallas's Intelligent Robotics and Vision Lab released VLA-Replica, a low-cost, easily reproducible real-world benchmark for evaluating vision-language-action (VLA) models, plus a public leaderboard.
- Built from off-the-shelf parts (SO-101 follower arm, light box, cameras), a user with no prior experience assembled it within one hour, enabling consistent policy evaluation across labs worldwide.
- The benchmark spans 10 diverse manipulation tasks with a small demonstration dataset, covering both in-distribution and out-of-distribution evaluation protocols.
- Fine-tuned with only 50 demos, NVIDIA's GR00T N1.7 VLA model performed on par with π₀ in ID settings and close to π₀.₅ in OOD scenes. Paper, code, checkpoints, and setup guide are available; accepted at the RSS 2026 Data-Centric Robotics Workshop.
More from Embodied
- Snap teams with Salesforce, Amazon and Nvidia to push Specs glasses into the enterprise — matt_slotnick · 2026-09-20
- Innate bot demo: founder shows robot running, dubbed 'Apple II of robots' — Scobleizer · 2026-09-20
- xTerra Robotics unveils DHAV-Hy, a lightweight wheeled-legged quadruped for security and inspection — Scobleizer · 2026-09-20
- NVIDIA GR00T N1.7 fine-tuned with just 50 demos beats pi0.5 by ~10% on VLA-Replica — YuXiang_IRVL · 2026-09-20
- Jev device review: fast and cheap, but C- at generalization, needs GPT-3-era prompting tricks — intellectronica · 2026-09-20
- UBTECH opens smart factory where humanoid robots build humanoids, one every 10 minutes — LexiLove · 2026-09-20