Does Imitation Learning Preserve Temporal Robustness in Dexterous Manipulation? An Expert-Learner Comparison Across Task Execution Speeds
Clinton Enwerem, John S. Baras, Calin Belta
cs.RO, cs.LG
2026-09-01
Maryland compares a scripted expert and ACT on ParcelStow: both hit 100% at nominal speed, then 84% vs 53% at 2×. Of ACT's 47 failures, 35 are insertion misalignments; none of 414 acquisitions without force closure finish the task.
Robustness tests for imitation learners usually change scenes, objects, or instructions, not how fast the task runs. In contact-rich manipulation, speeding up changes velocities, accelerations, tracking load, and contact transients, and success moves with them. Matching the expert at nominal speed does not mean the learner inherited the expert's dependence on speed. Prior work mostly evaluates execution faster than the demonstrations. This paper asks a narrower question: inside the demonstrated speed range, under matched initial conditions, how far apart are expert and learner success.
ParcelStow is the benchmark. A fixed-base humanoid grasps a parcel, reorients it in free space, and inserts it into a receptacle. A scripted expert and ACT, Diffusion Policy, and DAgger trained from its demonstrations are scored under the same geometry, the same success rule, and the same draws of initial conditions. Demonstrations cover speedup factors for the phases after acquisition. Nominal speed is r=1, the demonstrated range runs to r=2, then the grid extrapolates to r=3. One hundred episodes per cell. Success requires the parcel to settle upright inside a 10° tolerance.
Failures are read three ways. Stage latches mark whether an episode died at grasp, reorient, or insert. A relative-motion handoff starts from the learner's acquired pose and finishes the rest with the expert's hand–parcel relative motion, to rule out dropping the parcel in free space. Force closure at acquisition uses the physical threshold ε≤0 with no fitted parameters, as a candidate necessary condition for later success.
The expert and ACT-A both finish 100 of 100 at r=1. At r=1.5 the counts are 99 and 91. At r=2 they are 84 and 53, with Wilson intervals [0.76, 0.90] and [0.43, 0.63]. A paired bootstrap puts a 95% interval of [0.18, 0.44] on the difference. From r=1 to r=2 the expert drops 16 points and ACT-A drops 47. Two other ACT seeds start at 0.70 and 0.62 nominal success and still drop 34 and 48 points over the same interval, all larger than the expert's 16. Diffusion Policy records 0.68, 0.73, 0.54, not monotone in r. DAgger never leaves the floor, at most 3 of 100, so speed sensitivity cannot be discussed.
Growing the demonstration set from 50 to 100 to 297 lifts nominal success 0.27 to 0.44 to 1.00. At r=2 the same policies score 0.46, 0.44, 0.53, intervals overlapping. More data fixes the nominal number and does not fix the fast one.
Of ACT-A's 47 failures at r=2, 35 are insertion misalignments, 10 insertion jams, one excessive slip, one timeout. Under the relative-motion handoff every ACT-A acquisition keeps the parcel through reorientation and free-space transfer, yet only 64% finish the task, against 95% after expert acquisition on the same motion. None of 414 acquisitions without force closure succeed, across policies and speeds; the Wilson upper bound is 0.009. Acquisitions without closure are 8%, 21%, and 34% of ACT-A/B/C episodes. The continuous contact margin is a weak ranker: held-out AUROC is 0.62 at acquisition.
Nominal task success is not an acceptance test for imitation learning, at least not in contact-rich work where speed can change. ACT predicts action chunks with temporal ensembling; the expert emits a command every control step. The paper never sweeps chunk horizon, so chunking is not convicted. What it does show: temporal robustness has to be measured inside the demonstrated speed range, expert and learner on the same initial conditions, not as a single nominal accuracy. Force closure is necessary for success here and not sufficient.
For anyone shipping dexterous insertion, the eval sheet needs an execution-speed column, and alignment error at insertion is the stage to watch.
One task, state observations, fixed-friction simulation, a fixed-base humanoid, no hardware. Chunk horizon, temporal ensembling, and replanning frequency are not ablated, so the mechanism behind the expert–learner gap is still open: handoff pose error, tracking error, hand actuation, and receptacle contact were not separated. Pose offsets at failed versus successful handoffs overlap a lot (medians 3.1° vs 2.2°, 5.5 mm vs 3.9 mm); handoff state alone does not explain the failures. Demonstration sets larger than 297 were not tried. DAgger fails too hard to serve as a counterexample for speed robustness. It is a policy that did not learn the task.