RISE: Adaptive Imagination for World Action Models
Hongbo Lu, Liang Yao, Chenghao He, Hao Han, Fan Liu, Wenlong Liao, Tao He, Pai Peng
cs.CV
2026-08-20
RISE gates world-model imagination with planning gain versus cost. NAVSIM: 91.5 PDMS, 90.8 EPDMS, 2.4 rollout steps; the scheduler also plugs into DAWN.
World Action Models imagine future evolution before they act, yet most of them assign every scene the same imagination budget. Extra rollout on an empty road only adds latency; too little rollout at a messy intersection hides risk. RISE treats the missing piece as a sequential stop signal: is the current prefix enough to plan, and is one more imagined step worth the cost.
RISE is a plug-in scheduler on an Encoder–Predictor–Planner WAM, with a Latent Evaluator and a Rollout Gate. At each step the Evaluator emits a risk profile of what the prefix already reveals, plus a Future Planning Gain profile for continuing. The Gate compares that gain with compute cost and chooses Roll or Stop. Roll appends one future latent; Stop sends the chosen prefix to the Planner once. The horizon, from zero to the dataset maximum, emerges from those binary calls. It is not picked in advance.
Supervision splits. All-horizon planning scores supply gain targets. Real logs contain only the future that happened, so safety-critical alternatives are scarce. CounterDrive fills that gap: Wan 2.7 generates 10-second 1080p counterfactual clips from NAVSIM and nuScenes key frames, then experts verify trajectory validity, incident onset, and causal category. After filtering, nuScenes has 2,432/511 train/test clips and NAVSIM 5,013/1,000. Training has three stages: predictor and a variable-prefix planner, then risk and gain, then a cost-aware Gate. The encoder is a frozen V-JEPA 2 ViT-L.
On NAVSIM v1, PDMS is 91.5 versus DriveFuture at 90.7. On v2, EPDMS is 90.8 versus DriveFuture at 89.9. On nuScenes, mean L2 is 0.31 m and collision rate 0.10. The scheduler alone lifts PDMS from 89.7 to 91.2; CounterDrive alone reaches 90.5; together 91.5. Against random stop and latent-margin stop, the scheduler averages 2.40 rollouts, 287 ms latency, and 90.8 EPDMS, versus 89.5 and 89.7. Fixed depth is not universal: 1,248 scenes prefer zero rollout, 4,036 prefer depth 3; for the zero-rollout group, going to depth 4 drops EPDMS from 89.9 to 88.4. Dropped onto DAWN without touching its predictor or planner, PDMS moves 89.1 to 90.3.
| Method | NAVSIM v1 PDMS | NAVSIM v2 EPDMS |
| DriveFuture | 90.7 | 89.9 |
| EponaV2 | 90.4 | 88.9 |
| RISE | 91.5 | 90.8 |
Imagination helps driving, but the budget should not be wired into the architecture. Turning "how much would one more step raise the planning score" into a sequential gate keeps quality while skipping useless rollout. Transfer onto DAWN suggests this is compute allocation, not a backbone-specific trick. CounterDrive lifts risk-ranking AUC from about 0.5 to 0.93–0.96, a reusable asset for safety-oriented world models. The headline gain is 0.8–0.9 PDMS points, an incremental planning-benchmark step.
All experiments are driving; the authors say other embodied domains are untested. CounterDrive does not one-to-one cover the NAVSIM train set, and Wan-generated then filtered clips still sit off real crash statistics. Maximum horizons are 4 steps on NAVSIM and 3 on nuScenes, a narrow adaptive range. The lead over the strongest baseline is under one point, with no significance test. Perception-free NAVSIM/nuScenes scores also sit far from onboard latency and closed-loop control.