Where Should Action Generation Begin? A Learnable Source Prior for Generative Robot Policies
Meipo Dai, Qiyuan Zhuang, He-Yang Xu, Ying-Jie Shuai, Yijun Wang, Qi Dou, Xiu-Shen Wei
cs.RO, cs.CV, cs.LG
2026-06-16
LeaP replaces N(0,I) with a proprioception-conditioned diagonal Gaussian, reaching 81.6% on 15 RoboTwin tasks (+25.5 pp vs no-prior) and 80.0% on a real Franka.
Diffusion and flow-matching policies start action generation from N(0, I), a draw that ignores joint angles, the gripper, and the scene. Flow matching learns a velocity field that carries a source sample along a straight path to the target action. Part of the integration budget is spent moving uninformed noise into the right region of action space.
A2A, VITA, and BridgePolicy replace that Gaussian with an observation-derived start: proprioceptive history, a visual latent, or a vision-state bridge trajectory. Each start is a point estimate. Source uncertainty is left unmodeled. Injecting fixed-variance noise into A2A's proprioception helped robustness and hurt nominal scores. Where generation should begin is still a default, not a design axis.
LeaP replaces N(0, I) with a proprioception-conditioned diagonal Gaussian over an action chunk, meaning H future steps generated at once. A two-layer MLP reads proprioception zs (joint angles, gripper) and emits a mean μ and a log-variance. Those parameters are shared across the H steps, so the prior only parks the chunk in a state-dependent neighborhood. Samples are reparameterized from standard noise ε into a differentiable z0.
The generator still sees vision plus state. Its architecture and solver do not change. Vision is kept out of the prior; adding it pushes source samples away from the target action.
Lflow regresses the path velocity a − z0 and backprops into the prior. LNLL treats expert actions as samples from the diagonal Gaussian, with residuals weighted by predicted precision. Lalign is a CLIP-style contrastive pairing of each source sample with its expert action. Executing μ as the action scores 0%.
Fifteen bimanual RoboTwin 2.0 tasks, 50 demos and 100 rollouts per task, three seeds. Baselines: A2A, VITA, architecture-matched NoPrior, and a reimplementation of BridgePolicy. Encoder: DP3 PointNet throughout.
| Method | 15-task mean success |
| LeaP | 81.6±0.4% |
| VITA | 75.1±0.9% |
| A2A | 73.8±2.2% |
| BridgePolicy | 61.4±1.3% |
| NoPrior | 56.1±2.5% |
That is +6.5 points vs VITA and +25.5 vs NoPrior. NoPrior changes only the source.
The model is 13.22M parameters, 0.21M of them in the prior head. A2A and VITA use 28% more parameters and score lower; ACT, DP3, and BridgePolicy use backbones 6 to 20 times larger and still trail. On open laptop, LeaP reaches 91% by epoch 1000 and holds near 90%; the baselines remain lower at epoch 3000, and VITA falls from 61% to 35%.
On a Franka, three tasks, 20 rollouts each: LeaP 80.0%, A2A 68.3%, VITA 56.7%, NoPrior 46.7%.
Ablations, three tasks, one seed. State-only prior: 85.3%. Vision in the prior: 15 to 26 points worse. The diagonal Gaussian beats mean-only (78.0%), MSE (74.7%), fixed σ=1 (62.7%), full covariance, and a GMM. Dropping Lalign or LNLL lands near 74%; dropping both leaves 59.7%. The same prior lifts flow matching from 47.7% to 85.3% and a diffusion bridge from 68.7% to 76.7%. A bridge from N(0, I) scores 0%, so that run uses μ as the smallest viable source.
The source distribution is a cheap independent axis. Leave the generator and solver as they are, add a 0.21M head, and success rises in sim and on the robot, with faster convergence. Both generator families improve. A deterministic informed start already beats N(0, I); learning a state-adaptive variance beats that again; fixed-variance noise reverses the gain. On tabletop manipulation this is an inexpensive incremental lever. Do not assume the same margin on VLAs or dexterous hands.
A diagonal Gaussian ignores correlations across action dimensions. Full covariance was tried and did not help; cheaper structured covariances remain untested. Evaluation is tabletop only. Ablations use three tasks and one seed. BridgePolicy was reimplemented. Streaming Flow Policy was omitted because it has no chunk-level source. Real-robot evals use 20 rollouts per task with no reported variance. Vision in the prior shifts the source away from the target; the mechanism is not isolated. The 15-task mean is tight; per-task scores are not. On stack blocks two, LeaP itself is only 33.7%.