Learning What Actions Change: Action Effect Matching for Dynamics-Faithful World Models
John Won, Yonghoon Dong, Jinwoo Shin
cs.RO, cs.CV
2026-11-19
AFFECT injects logged noise into play-policy actions and matches perturbation-video covariances, lifting DROID policy-success prediction from 0.47 to 0.79 Pearson.
Robotics world models (RWMs) post-train video foundation models to generate future video conditioned on robot actions, and are increasingly used to evaluate policies and plan manipulation. A growing set of benchmarks (MiraBench, RoboWM-Bench) documents the same flaw: the models largely ignore their action condition. From one initial state, a slight change in reach decides whether the gripper grasps an object, misses it, or pushes it away, yet the model renders nearly identical plausible videos for all three. That breaks policy evaluation, since success predictions stop tracking reality, and misleads planners with visually smooth but physically wrong imaginations.
The paper, from KAIST and RLWRLD, reframes action sensitivity as a distributional problem: an RWM should learn how the distribution of future observations shifts as actions vary in a given context, not just render one realistic future.
AFFECT is a plug-in auxiliary loss with three parts.
Data first: perturbation play. A play policy (GR00T-N1.5 with the FLARE objective, trained on 56k frames, about 100 minutes, of task-agnostic teleoperated play) rolls out autonomously on the real robot. Before execution, temporally correlated noise is injected into each action chunk: white noise drawn per episode, smoothed with a 4-step Gaussian filter so the arm drifts instead of jittering in place, then rescaled by a declared per-dimension scale; grippers are left unperturbed. The pre-clipping noise is logged. Scenes are never reset; coverage grows by drift, and failed grasps and collisions are kept because they are exactly the action-dependent outcomes the model must learn. No repeated executions from the same start, no outcome labels.
The loss: matching action effects. Two frozen encoders reduce the perturbation and the video to small sets of scalar features. Perturbation features are hand statistics such as the signed sum of noise over a window and the squared sum minus its expectation; video features are statistics of the model's own latent, including channel-mean changes, fixed random projections, and temporal-difference summaries (54 on the Franka setup, paired with 16 perturbation features into 864 tests). Perturbation features are centered by subtracting an expectation computed in closed form from the declared noise parameters. The action effect is the expected product of a centered perturbation feature and a video feature, which is exactly their conditional covariance within a scene: it measures whether and how strongly a video feature moves when the perturbation runs larger. AFFECT makes generated futures reproduce the recorded covariances, pair by pair, squared and summed. Statistically this is a finite-feature MMD.
The design choices carry the paper:
Two implementation traps. The generated future must be sampled from fresh noise with a 2-step differentiable Euler sampler: reusing the flow loss's clean prediction fails because it is built from the recorded future and matches it whether or not the action is used, and a single step returns the conditional mean rather than a sample, matching only mean effects. The term fires every 8 iterations with weight calibrated to about 15% of the flow loss. The minibatch estimator is a U-statistic over independent units (episodes); squaring a naive batch mean would penalize generated variety and push toward mode collapse.
The headline metric is action fidelity: the Pearson correlation between policy success rates measured on the real robot and inside closed-loop world-model rollouts. All six backbone-embodiment combinations improve.
| Setting | Backbone | Pearson (base → +AFFECT) | FVD (base → +AFFECT) |
| DROID (real) | DreamDojo | 0.468→0.790 | 807→622 (-23%) |
| DROID (real) | Cosmos3 | 0.425→0.583 | 910→744 (-18%) |
| ALOHA-2 (real) | Cosmos3 | 0.298→0.694 | 333→257 |
| ALOHA-2 (real) | DreamDojo | 0.325→0.490 | 183→188 |
| UR5e (sim) | DreamDojo | 0.451→0.686 | 85→73 (short) |
| UR5e (sim) | Cosmos3 | 0.091→0.334 | 470→463 (short) |
Most visual metrics improve alongside, with gains on every metric at both horizons in simulation. Planning improves too: on five UR5e pick-and-place tasks, average success goes from 46.7% (random selection) to 60.0% (DreamDojo) to 68.3% (DreamDojo + AFFECT), a further 8.3 points; on a three-step ALOHA-2 long-horizon task the gain averages 8.4 points.
Ablations pin the mechanism: shuffling perturbation features across episodes (placebo matching) degrades FVD from 256 to 460, and unconditional feature matching gives 451, both worse than no auxiliary loss at all; training on 20% of the data shrinks the gain, so the benefit scales with perturbation coverage. A localization analysis shows frozen-feature responses concentrate in cells traversed by the arm, cube, or shadow, at 1.6 to 2.2 times the shuffled baseline, and stay at chance on the static table.
For anyone using world models for policy evaluation or planning on real robots, action fidelity is the gate, and this paper offers a cheap way through: no architecture change, no success labels, no repeated executions, 5 to 10 hours of perturbation-play data, one auxiliary loss, verified on two backbones (DreamDojo 2B, Cosmos3-Edge 4B) across three embodiments. The covariance-matching idea also travels: any generator whose response to a conditioning variable should match reality can borrow it. Keep the scale honest, though. This is a steady auxiliary-loss improvement, not a new paradigm, and one of the six FVD numbers moves the wrong way.
Stated by the authors: AFFECT only constrains responses its hand-designed feature bank can detect, and channel means plus random projections of the latent may miss fine object-pose changes; the covariance is a variance-weighted average, so overreaction in some contexts can partly cancel underreaction in others; and the play policy sees all cameras plus proprioception while the world model sees a single-camera window, so the auxiliary optimum can drift from the flow-loss optimum.
Questions from reading: success labels are binary judgments by human evaluators over 12 episodes × 4 checkpoints × 2 tasks per embodiment, a small and partly subjective sample. Ablations run on Cosmos3 at 10k iterations, short of the main-table training budget. Planning gains are measured inside a proposal-selection scheme (a 5-checkpoint ensemble plus a DINOv2 success estimator); how much survives with a single deployed policy is untested. The comparison with concurrent counterfactual-consistency work stays qualitative, and the CoCo baseline was only half-reproduced because its ASC2 component does not fit joint-space actions.