Energy score, not velocity matching, lifts LIBERO-90 from 78.9% to 85.2%

HINT-WAM: Hindsight-Informed Training for World Action Models

Phuong Tuan Dat, Qi Li, Xinchao Wang

cs.RO

2026-11-19

HINT-WAM distills a student's own denoising path against a teacher that sees the demo's future video, raising LIBERO-90 success from 78.9% (SFT) to 85.2% with no simulator.

What problem this solves

A world action model generates the robot action and the future frames in one pass. Fast-WAM's action expert is the small half of that stack. It emits a continuous chunk by integrating a velocity field from noise to a clean command. Dropped onto the 90 unseen LIBERO-90 tasks, the public checkpoint succeeds 15.56% of the time.

Supervised fine-tuning can copy the demonstrated action. Flow matching only ever trains on that action with noise added. The intermediate noisy actions along the sampler's own integral never show up in the loss. At deployment the error accumulates on that self-produced path. One observation also has several reasonable futures, and a single regression target does not identify which future the demonstrator took.

On-policy reinforcement learning does train on the policy's own behavior. Success in manipulation is usually a rare bit, so exploration and credit assignment stay expensive. Making the reward dense still returns a scalar. HINT-WAM supervises an action distribution on the noise states the student sampler visits. Observations stay inside the demonstrations. There is no simulator step and no environment reward.

Method

The base model is the released Fast-WAM. The 10-step Euler sampler and the 32-step action chunks stay as they are. A video expert and an action expert share one attention operation in every layer. The video expert reads only the current frame, and the integral then runs the action expert alone. Action tokens never see future frames. The observation is the camera image, the language instruction, and proprioception. The video expert is frozen throughout.

Without the future video the teacher's distribution is multimodal. Regressing its velocity field drives the student to the conditional mean, which can land between modes and match none of them. The energy score draws M implied actions from the student, a clean action read from the velocity field in one forward pass. One term pulls those samples toward a single teacher action. Another pushes them apart. Gradients never enter the rollout. They recompute the student only at the high-weight noise levels. The reported run uses K=1 and M=4, with weights proportional to the squared step size, so the mass sits near the end of the trajectory. Distillation by itself would also copy the teacher's mistakes, so an equal behavior-cloning term is computed on the demonstrations with no hindsight. Deployment loads the student alone.

Results

Every task is scored on 50 trials from fixed initial states. RoboTwin 2.0 is dual-arm manipulation. Evaluation uses 25 hard tasks picked before any training, 50 episodes clean and 50 randomized. The released checkpoint had already been trained on that data. The runs use 8 A5000 GPUs and a batch of 192. Stages A and B resume from SFT step 3,000. Hyperparameters are not retuned for RoboTwin.

MethodLIBERO-90RoboTwin cleanRoboTwin random
Fast-WAM15.56%86.48%83.44%
SFT78.88%not runnot run
Full SFT81.24%84.12%83.24%
BC control71.56%85.36%82.08%
Self-imitation79.62%not runnot run
Shuffled hindsight80.28%not runnot run
HINT-WAM85.18%87.04%83.68%

HINT-WAM is 6.30 points above SFT (paired t = 3.58) and 3.94 above full SFT, which updates the video expert too. Matched extra behavior cloning falls to 71.56%. Self-imitation, cloning the policy's own successes, beats SFT by 0.74. Shuffled hindsight, the same pipeline with another episode's future, reaches 80.28%, 4.90 points lower (t = 2.76). On RoboTwin, further cloning loses 1.12 points in the clean setting and 1.36 in the randomized setting against the released checkpoint. HINT-WAM beats that control by 1.68 clean (t = 2.00) and 1.60 randomized, and sits slightly above the checkpoint itself.

SettingNew tasksOld suites, avgOld long-horizon
No replay, SFT78.88%37.45%4.6%
No replay, full SFT81.24%32.85%2.2%
No replay, HINT-WAM85.18%44.05%8.2%
Replay, SFT76.70%91.55%81.2%
Replay, HINT-WAM81.56%95.10%90.2%
Released checkpoint15.56%96.55%94.0%

Without replay the four old suites fall from 96.55% to below 45% for every method. HINT-WAM's 44.05% is the highest average, and the long-horizon suite is still 8.2%. With replay mixed into every stage, old-suite success returns to 95.10%, within 1.5 points of the original 96.55%. Long-horizon reaches 90.2%, 9.0 points above SFT. New-task success is 81.56%, 4.86 above SFT's 76.70% (t = 2.66).

Another 22 hours of SFT, step 3,000 to 5,000, leaves success unchanged. The hindsight teacher takes 4.4 hours and distillation 34. The full pipeline is 1.28 times SFT and 0.56 times full SFT, which costs 127 hours.

With the teacher, the states, and the budget held fixed, velocity regression scores 79.02% (0.14 above SFT, t = 0.09), kernel MMD 80.63%, action-space MSE 82.93%, and the energy score 85.18%. That is 6.16 points over velocity matching, 48 wins to 17 losses (sign test p = 0.0004). Cloning alone scores 71.6%. Distillation alone scores 74.4%. Together they score 85.2%. Moving M from 1 to 4 raises success from 81.4% to 85.2%, and the curve is not flat yet. Supervision on noised demonstration actions scores 80.7%. Supervision on the student's own denoising states scores 85.2%.

Why it matters

Imitation is enough to take unseen LIBERO-90 tasks from 15.56% to just under 80%. More imitation overfits. Reinforcement learning still needs a reward and a search. The teacher in HINT-WAM is the student plus a small hindsight branch, so nobody trains a second, larger model. Only the action expert is updated. Full SFT spends 127 hours and still finishes lower.

On tasks the checkpoint has already seen, the gain is mostly less overfitting. RoboTwin moves by one to two points. The improvement is incremental.

The energy score is the piece that stands apart from the robot benchmark. Continuous actions have no convenient likelihood, and velocity regression averages across modes the student cannot tell apart. Here that difference is about six points.

Limitations

The on-policy states are states of the action sampler, not states of a closed-loop rollout. Contact failures and a shifted camera view never enter training. The image is always the current frame stored in the demonstration.

Forgetting without replay is still severe. Old suites drop from 96.55% to 44.05%, and the long-horizon suite to 8.2%. HINT-WAM forgets less than SFT. The pretrained skills are not preserved.

The clean RoboTwin comparison has a paired t of 2.00. The margin is small and mostly says that extra imitation overfits. All numbers are in simulation. There is no real-robot result, and nothing tests a new camera or a new arm.

The teacher is sometimes wrong, which is why the behavior-cloning anchor exists. Both weights are fixed at 1 and were not retuned on the second benchmark. M=4 has not saturated, and K=1 supervises only the single highest-weight point on the denoising path. Self-imitation adds successful rollouts from randomized placements. That is a different data budget from refusing new environment states, so the 5.56-point gap is not an equal-interaction comparison.

Terms

Source

Related papers

All paper explainers