VLA-Replica: low-cost real-world robot benchmark, MolmoAct2 beats π0.5 by 10% OOD

YuXiang_IRVL · x · 2026-09-16

• UT Dallas's Intelligent Robotics and Vision Lab released VLA-Replica, a low-cost, easily reproducible real-world benchmark for evaluating VLA models, presented at the RSS 2026 Data-Centric Robotics Workshop.
• Built from off-the-shelf parts (SO-101 follower arm, light box, cameras), the setup can be assembled in about an hour by someone with no prior experience and replicated across labs, with consistent in-distribution and OOD evaluation protocols.
• First result: MolmoAct2, fine-tuned with only 50 demonstrations per task on the SO-101 setup, outperforms π0.5 on OOD tasks by 10%.
• The team plans to keep benchmarking frontier models for real-world manipulation, with GPT-6 Astra next; paper, code, scenes, leaderboard, and training data are open.

Original post →

More from Embodied

• Teaching a robotic hand to walk on its fingertips, no robot arm required — Scobleizer · 2026-09-16
• Apple Weighs M8-Based AI Server With Nvidia NVLink to Reenter Server Market — gappyvalley · 2026-09-16
• Open-source companion robot Lamp gains environmental sensing: CO2 and humidity on Lamp Pro — dee_hw · 2026-09-16
• $20,000 Once: The Humanoid Robot Math That Ends Night-Shift Wages — cen6wkf · 2026-09-16
• Tesla logs 460,000 km testing FSD Supervised across Spain over two years — yunta_tsai · 2026-09-16
• SSAD 2026: dedicated reward models for crashes and sim-driven failure analysis in driving — abursuc · 2026-09-16

Embodied channel →