DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation
Fenghao Lei, Zhixiong Huang, Long Yang, Jiabao Chen, Peilin Huang, Han Fu, Zhuo Li, Xiaoxue Ren
cs.RO, cs.AI, cs.CV, cs.LG
2026-08-23
DELE-w0.5 predicts a future latent rather than generating video, then emits actions. On 640 real trials it hits 62.5% full-task success, 32.5 points above the strongest baseline.
A recent line of World-Action Models (WAMs) hangs robot control on a video generator. The model jointly synthesizes a future visual trajectory and an action sequence. The video prior brings spatiotemporal structure. Control, though, needs the physical state after the action, not a movie of the transition. Intermediate frames are high-dimensional, appearance-sensitive, and they force the control loop through iterative denoising of tokens the robot will never execute. Distillation and sparse tokens cut cost. They keep the same surrogate objective.
Vision-Language-Action (VLA) models map current observations to actions more directly, and they do not require the policy to represent the physical consequence of those actions. This DeepLeap technical report tries a third formulation: predict a compact future latent that is causally tied to the action, and drop video generation from the control loop.
In probability form, a VLA estimates p(action | observation, proprioception, language). A video WAM estimates p(action, future video | ...). DELE-w0.5 estimates p(action, observation at a future time | ...), with the in-between frames discarded.
The network is a two-stream Transformer. The condition stream encodes language with Qwen3 and three camera views with DINOv3 (head plus two wrists), projects both to hidden size 1024, concatenates them through a single-stream attention block, and writes the condition into QKV via AdaLN. The noise stream interpolates the action chunk and the future latent toward Gaussian noise, the flow-matching schedule, and produces its own QKV. Concatenated streams go through masked joint attention. The training loss regresses two velocity fields, L = La + λ Lo.
Action chunks are length H = 60. Single-arm actions are 7-DoF (relative xyz, Euler angles, gripper); dual-arm is 14-DoF.
The attention mask is more informative than the slogan. Language and current observation attend to each other and cannot see actions or future states, so targets do not leak into the condition. Action tokens see the condition and other action tokens, and they cannot see the future state. Future-state tokens see everything, which trains the mapping from instruction, current observation, and action onto the next physical latent. At inference the future-state groups are removed. The model emits actions only.
So the deployed policy does not generate a future state and then decode an action from it. The future latent is an auxiliary training target that shapes the shared backbone. The paper's "inferring action from future latent state" line and this mask are not the same mechanism.
All evaluations run on one Astribot S1 dual-arm robot in fixed real scenes. Four long-horizon tasks: open a hinged door, retrieve a Pepsi from a refrigerator, add ice to a cup, load popcorn into a microwave and start heating. Twenty formal trials per method-task pair, 640 trials across eight methods. Each rollout has a 180-second budget, no human help, and unsafe termination counts as failure. Stages must complete in order. Every baseline trains on the same task data for an equivalent of three epochs.
| Method | Macro progress | Full-task success |
| DELE-w0.5 | 81.3% | 62.5% (50/80) |
| GWP0.5 | 61.3% | 26.3% |
| XR0 | 59.5% | 30.0% (24/80) |
| π0.5 | 50.6% | 15.0% |
| GR00T-N1.7 | 22.0% | 0.0% |
Against the strongest baseline on each metric, full-task success rises 32.5 points and macro progress 20.1 points. Per-task successes are 16/20 door, 13/20 Pepsi, 9/20 ice, 12/20 microwave. Ice is the longest sequence and the weakest result.
Median core-model latency on an RTX 4090 is 87.5 ms, excluding network, decode, IK, and robot execution. Mean completion times on successful rollouts: 16.66 s door, 45.92 s Pepsi, 46.27 s ice, 39.50 s microwave.
Stage-reach maps show most baselines dying when they must keep an already-achieved state while switching skill: door grasp without synchronized rotate-and-push; Pepsi dying at the handoff or door close; ice dying at scooping and pouring; microwave opening the appliance and then failing insertion or heating. DELE-w0.5 reaches late stages more often. Its remaining failures still cluster on object transfer, precise release, and the final condition.
Two intervention rollouts, not in the main table: if the microwave door is closed mid-task, the policy keeps the bag and reopens the door; if the opening is too narrow, it pushes the door with the bag instead of returning to the demonstrated handle motion. The authors call this goal-conditioned behavioral recomposition and refuse to treat it as a quantitative emergence claim.
If the thesis holds, video generation does not have to sit in the middle of a robot WAM. For teams that care about onboard latency, 87.5 ms core inference is closer to a control loop than synthesizing video and then reading an action out of it. Under matched data and epochs, the gap versus π0.5 and GWP0.5 is large.
Treat the paper as a company technical report, not a fully ablated method paper. The gain could come from the future-state auxiliary loss, the dual stream, DINOv3, chunk length, or data. Without ablations, "drop video" is not an isolated factor anyone can copy.
At inference the future-state tokens are gone, and during training the action tokens never attend to them. Joint training with an auxiliary target is not the same as inferring actions from a predicted future. The slogan and the mask disagree.
The study uses one robot, fixed scenes, and twenty trials per cell. Demonstration count, λ, parameter count, and pretraining data are unreported. Most baselines are VLAs. GWP0.5 is the closest WAM. A full video-generation WAM is not in the table, even though that is the objective the paper attacks.
Ice succeeds 45% of the time; long-horizon error still accumulates. The emergence note is qualitative. There is no simulation suite and no table for lighting or large pose shifts.