A-0: 16k hours of handheld demos, zero robot data, 78% on ten bimanual tasks vs 27% for best VLA

AIRoA-0: Long-Horizon, Precise, Cross-Embodiment Manipulation from Handheld Data

Yueh-Hua Wu, Kei Ota, Hanlong Li, Hayata Yuasa, Jason Wang, Kengo Ikeuchi, Kensuke Chiyoma, Makoto Sugiura, Masafumi Takahashi, Masatoshi Tateno, Naoaki Kanazawa, Nicola Scianca, Petr Khrapchenkov, Satoshi Otsubo, Takehiko Ohkawa, Takuya Okubo, Tatsuya Matsushima, Yuki Noguchi, Yuma Shin, Zixuan Zhong

cs.RO, cs.AI, cs.LG

2026-11-03

A-0 learns bimanual manipulation from 16k hours of handheld demos with zero robot data, averaging 78% on ten long-horizon tasks at an unseen site vs 27% for the best VLA baseline.

What problem this solves

Data is the bottleneck for robot manipulation. Teleoperation gives one hour of data per hour of robot time, so corpora scale with the number of robots you own. UMI proposed the alternative: a handheld gripper with a wrist camera and 6-DoF tracking that a person holds in each hand, so anyone can record demonstrations anywhere and the corpus scales with the number of people. The AIRoA team pushed this route to foundation-model scale, 16,151 hours recorded with their YUBI gripper, and trained a bimanual system, A-0, on it with no robot data, no teleoperation and no RL at any stage. The robot appears only at test time.

Handheld data is also exactly the kind that is hard to learn from. Nothing in it shows a robot failing or recovering. The policy sees only through two wrist cameras and never the whole workspace. Hundreds of operators recorded over a hundred tasks in their own environments, and none of those environments is the evaluation site. A-0 is built to work under all of these conditions at once.

Method

Four layers, each handling the difficulty that arises there.

Data and coordinates. YUBI is finger-driven, carries a wrist camera and is tracked in six degrees of freedom. The corpus holds 16,151 hours and 10.65 million subtask segments (5.5 s on average), each labelled at two levels: a task instruction and a subtask instruction naming the step in progress. Both state and action are defined at the grippers themselves. An action is a target pose relative to the gripper's pose at the observation (20 numbers); the state is one gripper's pose in the other's frame plus both openings (11 numbers). Nothing refers to a robot base, a joint or a world frame, which is what lets human recordings and four different robot arms share one policy.

Pre-training. The backbone is the public π0.5 checkpoint (SigLIP encoder, Gemma 2B VLM, 300M flow-matching action expert), trained on the corpus for 300k steps at batch 2,048. The objective follows S2: conditioning on either level of language, plus a visual evidence budget gate that scores every image token with a keep probability and holds the mean to a target, with no annotation of where to look. Visualisations show the gate's weight landing on the grippers and the objects in contact while the table and background recede. An auxiliary loss asks the model to name the current subtask in text when given the task instruction, forcing the backbone to encode task progress.

Post-training and inference. A stage head classifies which stage of a long task is current and feeds it to the action expert as text, resolving the ambiguity where two wrist views are consistent with several steps. A proprioceptive memory compresses the last ten states, with timestamps, into four tokens added to the prompt. At run time the harness smooths the stage prediction with an EMA and decodes actions by non-uniform streaming: a two-slot buffer at noise levels (0.1, 1), where a chunk takes its large denoising step on entry and only a small correction on the newest observation just before it executes. One query takes about 60 ms on an H200.

Results

Ten bimanual tasks on a Franka at a site absent from the training data, 15 trials each, partial credit for partial completion. The four baselines (π0.5, Xiaomi XR-1, GR00T N1.7, MolmoAct2) start from their public checkpoints, are post-trained on the same task demonstrations, and get their best setting per task over a search of action horizons, frequencies and update counts. A-0 uses one recipe for all ten.

ModelAverage success, 10 tasks
A-078%
π0.527%
XR-123%
GR00T N1.716%
MolmoAct214%

A-0 ranks first on every task: 97% on colour sorting, 93% on the fuel bracket, 90% on the orange pyramid. On the robot-vacuum servicing task, two minutes of sequential work from one instruction, it scores 73% against 1% for π0.5. The weakest is earbuds at 30%. The gap narrows on the cup pyramid, where π0.5 reaches 71% against A-0's 80%. Six of the ten tasks needed three to six hours of demonstrations for post-training, which takes under four hours on eight H200s. A scaling study fits a power law L = 0.063·D^-0.035 (R² = 0.98): from 1,000 to 16,000 hours, each doubling of pre-training data lowers post-training loss by about 2.4%, and models pre-trained on less data never catch up after post-training. When a person pushes the bins aside mid-run, the arm redirects to the correct bin in one continuous motion. The same weights run unchanged on a TX-G2 humanoid, OpenArm and UR arms.

Why it matters

This is the handheld-collection route carried to VLA pre-training scale, with the full stack released: the dataset (claimed as the largest open handheld-gripper corpus with task and subtask instructions), pre-trained weights, the post-training recipe and the YUBI hardware design. The arithmetic for a robotics team is direct: a new task costs three to six hours of hand-recorded demonstrations plus four hours on one 8-GPU node, with zero robot time. Collection scales with people instead of robots, and any arm that mounts the gripper runs the same policy.

The 78-to-27 margin should not all be read as model quality. The baselines' checkpoints were built for scene cameras and robot data; wrist-only views and handheld trajectories sit outside their native regime, and interface mismatch accounts for part of the score.

Limitations

The paper has no separate limitations section; these are read out of it. The 78% includes partial credit, so it is not a strict full-task completion rate. The baseline mismatch stands even after per-task tuning. The scaling exponent is flat: doubling data buys 2.4% lower loss, and the mapping from loss to success rate is never verified. Earbuds at 30% show fine small objects remain hardest. The disturbance test covers one scenario, bins being pushed, and two wrist cameras rule out tasks that need a global view. Cross-embodiment has a hard precondition: the robot must take the YUBI gripper as its end effector.

Terms

Source

What people are saying

Related papers

All paper explainers