MotorMind: zero-shot robot manipulation with general VLMs, 95% success on real xArm6
UIUC-CS · hf · 2026-10-05
MotorMind from UIUC lets a general-purpose VLM operate a robot directly — no task-specific policy training, coding agents, or grounding tools like SAM3.
- A harness connects VLM-proposed mid-level actions to deterministic robot control and execution feedback, with asynchronous monitoring and background memory updates
- Zero-shot results: 66.7% success on LIBERO-PRO base suites and 53.8% under perturbations, versus at most 13.3%/19.2% for prior zero-shot methods
- On a real xArm6, the same interface averages 95% success across direct manipulation and human-perturbation settings
- Swapping in a stronger VLM backbone improves performance; remaining failures (visual grounding, embodied reasoning, action knowledge) shrink as VLMs improve
The result shows general VLMs can do effective zero-shot manipulation given the right mid-level action representation and execution harness.
More from Embodied
- Waymo rides hit papercut issues: stalling, jerky turns, near-miss reversals — henloitsjoyce · 2026-10-05
- DIY Muse Hardware: Muse Gadgets SDK Ships Prebuilt Firmware for ESP32 Devices — op7418 · 2026-10-05
- Half-Life: Alyx runs natively on Apple Vision Pro with no PC or streaming — banteg · 2026-10-05
- Tesla FSD fleet tops 15B real-world miles, logging 60 human lifetimes of driving per day — XFreeze · 2026-10-05
- Tianjin University Lab Unveils 3-Gram Non-Invasive Brain-Computer Interface That Hides in Hair — yogthos · 2026-10-05
- Huawei has mass-produced 381 τ chips; Hyundai to deploy 25,000 Atlas robots — 创业邦 · 2026-10-05