One Future, Every Robot: Label-Efficient Collective-State Prediction with Decentralized JEPA
Alan-Barsag Gazzaev, Alexey Gavrilov, Sergey Muravyov
ICRA 2027
cs.RO
2026-07-31
CS-JEPA has each robot predict one shared 1105-dimensional future latent from a 16-frame local history and 256 bytes per edge. It beats raw-future reconstruction on label efficiency across topology and size shifts, transferring from 18 to 108 robots.
No member of a robot swarm sees the whole picture, yet the swarm has to act coherently. Existing work splits into forecasting each robot's own trajectory, scoring state through a centralized critic, or reaching consensus on the present. This paper asks a different question: can every robot, from only its local observations and bandwidth-limited messages, predict the same future collective state of the whole swarm, and do it with little downstream labeling, so a handful of globally labeled episodes suffices to read the state out? The authors call this decentralized shared-state prediction.
The target is specific. It is not per-agent trajectory forecasting, not a centralized state estimate, not present-moment consensus. It is N local predictors each independently producing an estimate of one future quantity.
CS-JEPA (Collective-State JEPA). A JEPA predicts future representations in a learned latent space instead of reconstructing raw observations. The per-robot deployment contract is tight: a 16-frame local history, one 64-float (256-byte) recurrent message per directed edge per step, one synchronous round. No global pooling, no target encoder at runtime, the episode clock is zeroed, no future actions.
Key points:
Training and deployment are kept strictly separate. The privileged future target exists only in pretraining. At deployment the target encoder, tokenizer, decoder, and anchor head are discarded, the local encoder and recurrent predictor are frozen, and a small ridge probe is fit on 6, 12, or 24 globally labeled episodes.
The capacity accounting is honest: CS-JEPA has fewer trainable parameters than the Future-Recon baseline (which adds 9,607 training-only parameters through its raw-state decoder), and both deploy the same 123,713 representation parameters. A CS-JEPA win comes with less training capacity.
The main result is a prospectively registered five-seed follow-up (reconstruction-minus-JEPA error AUC; positive favors JEPA). In-distribution accuracy is +0.0046, ring topology +0.0572, mutual-kNN +0.0534, size-out-of-distribution +0.0425; agreement rises too (ring +0.0988). All five outer seeds are positive, every lower confidence bound is above zero, and CS-JEPA's curve sits below reconstruction at every label budget. The size split trains on 10 and 18 robots and tests on 36, 72, and 108, up to 6x larger, and still transfers.
A sealed eight-seed action-conditioned study drops four-step counterfactual value MSE from 0.0285 to 0.0155, a 45.5% reduction, and lifts within-context candidate-score Pearson correlation by 0.129 (0.350 to 0.479), favorable in 8 of 8 seeds at p=0.0078.
A 16-seed fully closed-loop study shows CS-JEPA planning beating nominal control on composite utility (+0.0105, p=0.0007), task score, and connectivity. Against Future-Recon it produces more consistent first-plan-coefficient choices (+0.1447, p=0.00006) and lower pairwise disagreement.
One thing must be stated plainly: against Future-Recon, composite utility does not differ (-0.0002, p=0.953). CS-JEPA wins on decision consistency, not on utility.
For multi-robot and swarm RL, this is a label-efficient, size-invariant predictive primitive that runs fully decentralized under a fixed, tiny message budget. A shared future target aligns each robot's local evidence into mutually compatible predictions, and it emerges without any explicit agreement or consensus loss. One frozen representation plus a small probe generalizes across topology shifts and 2-to-6x size shifts. If global sensing and labeling are expensive in your system, this trims some of that burden.
Read it honestly, though: CS-JEPA beats nominal control and beats reconstruction on prediction accuracy and decision consistency, but it does not beat reconstruction (or specialized controllers) on actual closed-loop utility. The win over the matched baseline is consistency, not performance.
The authors concede the study is simulator-only, assuming shared normalized coordinates and noiseless observations; hardware transfer, onboard latency, and safety are open. Pretraining uses privileged future swarm state. The five-seed target-isolation study covers only the evaluated graph families and swarm sizes. Action-conditioned training applies one shared plan across the swarm per branch, while decentralized execution can combine different plans, leaving a train-execution gap. The closed-loop evidence supports utility over nominal control and first-coefficient consistency, not utility superiority over Future-Recon or specialized controllers.
Open questions from reading it: the effect sizes are small (the utility delta is 0.0105; the 45.5% reduction is an MSE move from 0.0285 to 0.0155, real but within a narrow simulation regime). Tasks are three hand-designed ones (flocking, formation, coverage) on bounded small-world graphs of degree at most 4; generalization to messier real swarm tasks is untested. The title promises that every robot predicts the same future, and agreement does emerge, but it is not perfect: each robot still carries a different error at deployment.