HARP-Bench: learned robot policies score 6-19% on 37 tasks involving humans
gan_chuang · x · 2026-11-10
HARP-Bench, presented at CoRL 2026, is a benchmark and data generation framework for robot safety in human-present manipulation, arguing that existing robotic simulation datasets over-focus on tabletop manipulation and ignore working alongside humans.
It defines four modes of human involvement:
- Co-presence: robot works beside a person
- Contribution: human and robot share the work
- Interruption: robot waits then resumes when a person inspects objects
- Cueing: human actions set the task goal
Across 37 tasks, a full-state expert reading human intent scores 99.1%, but learned policies collapse: ACT 10.4%, π0 9.4%, SmolVLA just 5.7%. Even the best learned policy only reaches 18.8% on cueing tasks, showing current VLA policies badly fail to read and cooperate with human intent. Code, asset pack, and an interactive demo are available.
More from Embodied
- HermesEnabler2 confirmed working on Apple Watch Series 10, unlocking Hermès watch faces — gregbarbosa · 2026-11-10
- Fable 5.2 hits 64.9% on RoboDojo, beating purpose-built robot AI systems — CyberRobooo · 2026-11-10
- Caltech robot builds a structure in orbit autonomously, a first for space construction — GregCook2011 · 2026-11-10
- Robot paints original watercolors and ships them with telemetry — a one-man robot art club — repligate · 2026-11-10
- Day 5 of a DIY robot arm: CAD render of the differential wrist in progress — _Stocko_ · 2026-11-10
- SuperMimic's 30× gradient SNR edge over score-function cuts imagined steps 64× vs PPO — daibond_alpha · 2026-11-10