DroneWAM: Efficient World Action Model for Drone Visual Navigation
Liang Yao, Fan Liu, Hongbo Lu, Wei Xu, Jianyu Jiang, Yijun Shen, Chuanyi Zhang, Pai Peng
cs.CV
2026-09-27
DroneWAM predicts in JEPA latent space and sets imagination depth with a DPO Gate. Open-loop ATE drops 39.8% vs FastWAM; average depth falls from 8 to 4.58.
A drone navigates from partial egocentric frames. Each control command moves the airframe and changes the next image, so the agent has to anticipate where a candidate action will send both pose and appearance. World models learn those transitions. World-action models (WAMs) also emit the action. Onboard flight calls that prediction over and over under a tight compute budget.
Two existing lines miss the constraint. Vision-language navigation can ground a semantic goal, yet future pixels usually stay implicit. Aerial world models do spell out how the view evolves with motion, but they propagate visual features across many imagined steps and typically spend the same rollout depth on every scene. Straight flight over open ground and a turn between buildings get the same bill. Most aerial visual-navigation benchmarks also write flight as 4-DoF, translation plus yaw. Roll and pitch change the camera; 4-DoF never sees them.
An aerial WAM that can run on the vehicle therefore needs three properties at once: a prediction space that keeps scene structure and motion without paying for pixel reconstruction; a compact state, because that state is copied at every imagined step; and a rollout length that follows the scene.
DroneWAM predicts the future in representation space and never paints the next RGB frame. The backbone is JEPA (Joint-Embedding Predictive Architecture): it forecasts future embeddings rather than pixels. A frozen ViT-L emits dense visual tokens. A pretrained Resampler (TokenAE in the paper) then compresses 512 tokens to 128 latents before any multi-step imagination, so the cost that repeats at every step is already small. The goal image shares the same frozen encoder and Resampler.
The controller sees four RGB frames, matching 7-D motion states, executed-action history, and a goal image, and outputs an eight-step ego-frame 6-DoF action plan. Training has three stages.
At test time the model scores each extra step. It stops when the score clears the threshold, but it executes the first action of the highest-scoring plan seen so far, then replans from the new observation.
The dataset is DroneNav-6D, built in Unreal Engine 4.27 and AirSim across five simulated worlds, with synchronized RGB, 6-DoF pose, control commands, and random wind. After filtering, the corpus has 3,837 clips, 1,592,170 frames, and 63.03 hours at a median 7 Hz. Train/val is 3,453/384 clips.
Open-loop, every method gets the same observation context and no intermediate feedback. DroneWAM is lowest on trajectory error and on latency.
| Method | ATE↓ | Rel.ATE↓ | Latency (ms)↓ |
| V-JEPA 2-AC | 27.4317 | 0.6621 | 64,617.72 |
| DINO-WM | 9.0923 | 0.4459 | 29,622.24 |
| NWM | 3.7580 | 0.3740 | 403,827.78 |
| LeWM | 4.2259 | 0.4164 | 654.37 |
| FastWAM | 1.8366 | 0.1919 | 712.33 |
| DroneWAM | 1.1062 | 0.0578 | 483.29 |
Against the strongest baseline, FastWAM, ATE falls from 1.8366 to 1.1062 (39.8%) and latency from 712.33 ms to 483.29 ms (32.2%). NWM paints future pixels; its 400k-ms open-loop latency mainly shows that explicit image generation is not an onboard option.
Closed-loop, the model replans from new frames. Navigation progress rises from FastWAM's 0.510 to 0.774, ATE from 119.8063 to 83.4461, MSE from 124.30 to 91.96, latency still 483.29 ms. The paper states that none of the evaluated methods finishes an episode, so the system is not ready to fly. On a separate easy set of 50 mostly straight routes with mild turns, DroneWAM reaches 88% success against 74% (NWM) and 72% (FastWAM), progress 0.95, latency 460 ms. Hard routes fail completely; easy routes can be flown.
Ablations separate space from time. Adding the Planner drops Rel.ATE from 0.6621 to 0.0584 and latency from 64,617.72 ms to 656.12 ms. The Resampler then cuts latency to 582.65 ms with almost no error change. Adaptive rollout continues to 483.29 ms at Rel.ATE 0.0578. Compressing 512 tokens to 128 cuts latency 18.3% (591.28 to 483.29 ms) while Rel.ATE moves from 0.0574 to 0.0578. At 16 tokens Rel.ATE rises to 0.0659.
Fixed depth 8 has Rel.ATE 0.0688. Random stopping at mean depth 4.581 raises it to 0.0722. The learned Gate at mean 4.578 reaches 0.0578. Same compute, random early stop is worse than a full rollout. Grouping samples by the depth that minimizes error, the best depth spreads across 1 through 8 rather than piling at the maximum.
The same Gate on the LIBERO manipulation benchmark, versus fixed 8 steps, cuts latency from 890.8 ms to 411.9 ms (53.8%), normalized action RMSE from 0.2685 to 0.2652, gripper accuracy from 89.83% to 90.35%, at mean depth 3.56. All four suites beat fixed-8.
This is a drone WAM that treats imagination cost as a first-class constraint. JEPA keeps prediction in embeddings and skips RGB. The Resampler shortens the state that every step carries, and the Gate shortens the number of steps. 128 tokens, 4.58 steps on average, about 0.48 s open-loop: onboard prediction should spend compute per scene.
DroneNav-6D's 6-DoF pairs and the contract of variable-depth imagination then executing the first action of the best-scoring plan along the way can move to other aerial WAMs. It remains an incremental result in simulation. Closed-loop success on hard routes is zero. 88% on easy routes shows the loop can fly, not that it can fly a dense city.
The paper lists its own: a fixed token budget and a capped imagination horizon may need retuning across scenes and compute; quality is tied to what the pretrained encoder and Resampler keep; closed-loop success in complex scenes is very low; the system is not ready for real deployment and still needs higher success, more robustness, and tests on a physical platform.
The evaluation is almost entirely on their own simulator. In the open-loop table, DINO-WM, V-JEPA 2-AC, and NWM sit two to three orders of magnitude slower, so the real comparison is FastWAM and LeWM. The Gate stops too early: appendix Case 4 halts at depth 1 with Rel.ATE 0.2160, while depth 8 reaches 0.1285. Closed-loop success requires no collision, terminal position error within 2 m, and attitude error within 0.35 rad. Hard-route success is zero; progress 0.774 is easy to over-read. An appendix LIBERO variant that injects language lifts average success from 45.23% to 59.94%; that is extra modules, not the original DroneWAM. The 483 ms figure is batch-1 synchronized GPU wall-clock, excluding data loading and image preprocessing. There is no embedded-board number and no real flight.