See to Reach, Feel to Grasp: Learning A Blind Grasp Reflex for Anthropomorphic Robotic Hands
Alexander Alexiev, Tzu-Yuan Lin, Sang Min Kim, Ho Jae Lee, Yonghyeon Lee, Sangbae Kim
cs.RO
2026-09-25
A proprioceptive hand policy grasps without vision: 96% on YCB and 95% on 3,028 GraspXL objects, with grasp-score Spearman ρ=+0.77 vs unseen Ferrari-Canny ε.
Dexterous grasping is a contact-rich feedback problem. A common pipeline predicts a grasp pose from vision, moves the arm to a pre-grasp, then closes the fingers along a scripted trajectory. Residual perception error, a slightly wrong approach, or a nudged object leave that close-loop with almost no way to recover.
Many learned systems fold the arm and the hand into one vision policy, conditioned on point clouds, poses, or images. Global geometry helps with reaching. It also ties local contact control to object shape and scene layout, so a new arm controller or a moved object often means retraining. People reach by sight and finish the grasp by feel. The question here is whether a five-finger hand can stabilize contact with joint encoders only, and no vision at grasp time.
Three modules share a thin interface. The arm module commands a palm pose and a binary engage bit. The hand policy outputs 20 joint targets plus a scalar grasp score γ in [0, 1]. A low-level velocity controller turns those into actuator commands.
The hand is trained in two stages. A privileged PPO teacher sees per-link contact bits, the object position in the palm frame, and joint angles. A DAgger student copies the full action from a history of H=5 joint readings and the tracking residual (measured angle minus the last position command). Contact stalls a finger and leaves a persistent command-measurement gap; that history is how the student infers contact without tactile, force, torque, or object state at deployment.
Joint targets are blended toward a PCA manifold fit on human grasps, with β=0.5, so fingers stay coordinated but can leave the manifold to fix retargeting error. γ is an action dimension with no stability labels. During training the arm is a two-phase controller: keep reaching when γ is low, lift when it is high, and drop back to reaching if the score collapses. Lifting too early hurts the height reward; waiting too long delays it. The teacher learns the switch from those consequences.
Training runs in MuJoCo Warp with 4096 parallel worlds on the full YCB set plus scale, mass, pose, disturbance, and servo-gain randomization. The teacher is a 256-unit GRU plus MLP; the student is a (256, 128, 64) MLP. Both stages take about 4 GPU-hours on one L40S. Hardware is a 7-DoF Flexiv Rizon 4 with a 20-DoF Robotis five-finger hand, 1 kHz low-level control, 11.9 Hz policy.
Each method gets a 30-second window with re-grasp allowed. Success is holding the object within 10 cm of a raised target for 2 seconds.
| Method | YCB (78) | GraspXL (3028) | Dynamic | Pose generalization |
| Scripted close | 31% | 39% | 15% | 30% |
| RobustDexGrasp teacher | 98% | 97% | 8% | 15% |
| Ours, no grasp score | 73% | 82% | 75% | 70% |
| Ours | 96% | 95% | 92% | 90% |
The end-to-end point-cloud policy leads on static grasps and drops when the object moves or leaves the training spawn box. The frozen hand policy plus an arm that updates the palm target online covers more of the workspace. The learned score tracks a Ferrari-Canny ε force-closure margin computed from contacts the policy never sees: Spearman ρ̄ = +0.78 on successes, +0.77 overall.
Ablations: β=1 (pure manifold) falls to 28% success. Joint angles alone reach 91%; q−q^{tgt} reaches 60%; the tracking residual q−q^{cmd} is 96%. Doubling or halving stiffness stays at 95% and 93%. Halving max velocity drops success to 76% and stretches time-to-hold from 3.9 s to 9.1 s.
On hardware the same frozen policy picked up 29 objects of mixed shape and material. That is a diversity demo, not a success-rate study; there are no repeated trials. The hand also composed with unseen arm behaviors: RexOmni language pick-and-place, SAM2 tracking of a moving object, a scripted lightbulb screw, plus new mounts and other arms.
For dexterous manipulation, grasping becomes a reusable hand skill instead of a joint arm-hand vision policy. The arm still has to see and reach. The hand closes the loop after contact. The sensor list is short: stock joint encoders, no tactile array, and no tactile sim-to-real gap.
This is not a claim that robots can skip vision. If the palm never enters a graspable range, the reflex never starts. On static simulation it also does not beat a jointly trained teacher. What it buys is modularity and composition when the object or the arm controller changes.
The 29 real objects were not repeated, so they are not a success rate. The grasp score grew out of a reach-then-lift reward; it transferred to wrist rotations for a lightbulb, which does not mean it is calibrated for every post-grasp motion. The policy never sees contact geometry, so complex sliding or tiny objects may sit outside what joint history can express. The end-to-end baseline was trained on static poses, which makes the dynamic comparison uneven, though even with an updated point cloud it still scored 8%. There is no Limitations section; failure modes show up mainly in the ablations.