MASS: Multiplayer World Models with Authoritative Shared State
Ziqi Cai, Siqi Yang, Yimu Wang, Zixian Gao, Yunheng Liu, Shuchen Weng, Erwin Wu, Kaipeng Zhang, Boxin Shi
cs.CV, cs.HC
2026-08-07
Folding state into visual latents breaks multiplayer. MASS splits a Logic Engine from a Rendering Engine around one authoritative state, hitting 1,024 concurrent players with zero cross-view disagreement.
Video world models pack world state into view-dependent visual latents. That is fine for one camera, but in multiplayer it breaks: each player keeps a separate recurrent visual history that re-encodes the same shared content, wasting compute, letting simultaneous views disagree about the same entity, and tying simulation cost to spectator count.
MASS borrows an old trick from multiplayer netcode: an authoritative server advances state, dumb clients just render. It disentangles world dynamics from view rendering.
Two learned engines flank one authoritative shared state.
The Logic Engine is a decoder-only Transformer that advances a global, authoritative, typed state from joint player actions and exogenous inputs, once per tick. There is no hand-written transition function; the transition is learned. It processes tokenized records with schema-derived masks. This state is the system's sole recurrent memory and synchronization reference.
The Rendering Engine is a residual U-Net that renders an independent, consistent view for any requested camera on demand, reading the same shared state. Because rendering no longer feeds back into state, cameras can be added, moved, or upgraded without changing the world.
Simulation cost is now independent of view count. Better, entity persistence, position accuracy, and structural validity can be measured directly on the predicted state without rendering a frame first.
On a matched multiplayer Snake benchmark, MASS leads six of seven metrics. Cross-view disagreement is 0.000 against 0.984 to 1.000 for baselines; invalid-state rate 0.177 against 0.981 to 1.000. LPIPS is 0.098, best (the strong baseline B-UN scores 0.123), yet B-UN's parser-recoverable state rate collapses to 0, showing that matching pixels does not mean matching state.
Evaluating the Logic Engine directly: at one step, semantic accuracy is 97.9% and position 99.1%; by 128 steps position drops to 72.3% and full-state exact match to 0. As a contrast, dense-latent designs (Joint U-Net, independent-head CNN, RSSM) never exceed 2.7% position accuracy at any horizon, and every predicted tick is self-contradictory. The typed model reaches 99.1% position accuracy with zero contradiction. This is the paper's hardest result: state crammed into visual latents cannot back an authoritative state.
Rendering quality at 256x256: Snake 38.82 dB, Pac-Man 33.95, Frogger 40.24, Crate Pusher 28.50. For scale, it runs 1,024 concurrent players over 10,000 recurrent steps; the Logic Engine processes 5,121 records per tick (1,024 players plus 4,096 food buckets plus 1 global), and rendering goes from 189.6 ms for one view to 842.4 ms for 1,024.
Under server stalls, client prediction holds 1.000 in-view agreement with oracle joint inputs, but drops from 0.815 at k=1 to 0.429 at k=8 with local actions only.
The thesis is an architectural claim: world models that squeeze state into visual latents cannot do multiplayer. To scale, you need what real game engines already do, an explicit authoritative state with simulation and rendering separated. For teams building multi-agent world simulation, it is a concrete path to thousands of actors and thousands of cameras, and state correctness can be checked before a single frame is rendered.
The method is validated only on 2D games with declarative schemas (Snake, Breakout, tile-merger, Pac-Man, Frogger, crate pusher). 3D environments and richer entity interactions are explicitly future work.
The Logic Engine's position accuracy decays with horizon: 72.3% at 128 steps, with zero full-state exact match. Long rollouts drift. Rendering quality also varies sharply across games, with Crate Pusher at only 28.5 dB. These are in the paper's own data, but they temper how far the scalability claim travels in practice.