HumanCLAW open-sourced: all 9 SOTA VLMs fail embodied benchmark, best hits only 16.8%
liuziwei7 · x · 2026-08-18
HumanCLAW, an evaluation framework decoupling a VLM's action intelligence from low-level motor control, is now fully open-sourced. At every sub-second step, an off-the-shelf VLM issues an atomic whole-body skill realized as continuous full-body motion with real physical consequences (contact, collision, gravity), while balance and motor-tracking failures are factored out — isolating the model's moment-to-moment action decisions.
HumanCLAW-Bench provides 1,218 long-horizon egocentric find–navigate–interact episodes across 41 indoor scenes. None of nine SOTA VLMs solves it; the best reaches only 16.8% success, suggesting current VLMs lack embodied self-awareness.
More from Embodied
- CMU's Andrea Bajcsy on what safety means for generalist robots, beyond collision avoidance — CSProfKGD · 2026-10-16
- Codex drives a physical CAN-bus motor, eyeing a fully Codex-built robot — OpenAIDevs · 2026-10-16
- Berkeley paper uses the Midas myth to rethink robotics beyond human imitation — begusgasper · 2026-10-16
- Team uses Claude Opus 5.5 to reproduce core claims of 10 recent robotics papers end to end — teortaxesTex · 2026-10-16
- Kinex: a harness that connects coding agents to any physical robot for scalable learning — daibond_alpha · 2026-10-16
- Gemini is a robotics outlier: dropping it lifts capability-robotics correlation from 0.20 to 0.96 — ycombinator · 2026-10-16