Hybrid LLM agent scores 83.2%, beats nine VLA policies on new SO-101 robot arm benchmark
Truman Hickok · hf · 2026-10-28
Truman Hickok releases SO-101 Bench, a simulated benchmark for language-conditioned spatial manipulation built on an Isaac Lab digital twin of the low-cost SO-101 arm: 4 tasks, 138 objects in seen/unseen splits, 916 evaluation episodes, and 3,629 demonstrations. Across nine VLAs and world-action models, the best learned policies score near 61%, while an LLM agent that writes code and hands grasping to a learned policy scores 83.2%, with ablations attributing the lead to harness and model choice. Simulation ranks policies identically to the real arm (Spearman ρ=1.00) though it overestimates success.
More from Embodied
- RAX lets a real robot explore for 2 hours and turn learned skills into reusable code — no demos needed — du_yilun · 2026-10-28
- OpenTac tactile sensors demoed at CoRL 2026 among 17 robot demos — _krishna_murthy · 2026-10-28
- cixliv: robot fighting may be the first realized market where bots beat average humans — cixliv · 2026-10-28
- Can the Hiwonder MentorPi T1 run fully offline as a robot pet? Pi 5 questioned — oulu2006 · 2026-10-28
- Testing Mint Modeler to Build the Menlo Research Robot — freelerobot · 2026-10-28
- Khosla-backed startup builds a 'mind-reading' cap with 100,000 tiny chips to write AI prompts — Distinct-Question-16 · 2026-10-28