Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control
Mingyang Chen, Shengdong Chen, Xiaoxiao Fu, Bosheng Gong, Haoyuan Guo, Bowen Li, Jiawen Li, Kejun Li, Tianpeng Li, Yin Liu, Haoze Sun, Zeyang Tian, Meng Wang, Xinmiao Wu, Jiangqiao Yan, Zining Zhao
cs.CV, cs.LG
2026-09-16
A 5B world model with joint keyboard and text control streams 832×480 at 24 FPS for about $0.009/min, scoring 81.0 on 158 WBench Navigation cases.
A playable generated world has to keep three things in one session: explore, intervene, and act on the feedback. Interactive world models have made keyboard and camera control solid. Language-conditioned systems can change content and behavior. The two are often separate modes. Keys give continuous movement and view; text specifies characters, events, and the surroundings. Split them, and a user cannot rewrite an event while walking, then keep moving to see what changed.
The temporal scales also disagree. A text-directed event spans several generation blocks; online interaction needs short causal chunks that accept a new instruction mid-stream. Cost sits under both: people will only retry if 480p is real-time and cheap.
The backbone is Wan2.2-TI2V-5B plus a 3.68M-parameter action branch (0.074% of the 5B stack), zero-initialized at the projection. Actions are eight nonnegative strengths: WASD for movement, IJKL for view, combinable. The model conditions on keys directly, skipping camera pose as an intermediate. Pose increments fight generated motion over long rollouts, and text-driven view changes make that mismatch worse. Magnitudes are relative intensity, not meters, calibrated per source. Action features add to the matching latent frame's visual tokens. Text conditions its interval through cross-attention; a prompt update refreshes text K/V and keeps visual K/V.
Data mixes image-text, video-text, recorded gameplay, camera-annotated real video, and synthetic joint-control sequences. Filters drop bad exposure, blur, low aesthetics, and OCR-heavy frames. Pure W-forward clips are downsampled. Joint-control videos align segment captions and continuous strengths to one timeline and keep full two-event clips to supervise prompt switches. Thirty-second gameplay clips lengthen history.
Training has four stages. Bidirectional adaptation injects actions on short clips, mixed with action-free T2I/T2V/I2V, then 30-second videos. Autoregressive adaptation splits in two: a segment-level teacher jointly denoises each prompt interval (bidirectional inside, causal across) to learn continuation through prompt changes; a block-level student emits four latent frames (16 video frames) from current inputs. First-frame synthesis is trained as T2I then I2V, so a bad opening frame does not poison the rest. Student history is mixed clean, noisy, blurred, and rescaled around the latent-channel mean. ODE initialization plus local consistency distillation follows, with a causal teacher on ground-truth history so a bidirectional teacher cannot leak the future. DMD then trains the student on its own rollouts, with the segment-level branch as the real scorer. Variable-length samples use a no-grad rollout and a packed replay, as in Self Gradient Forcing. A subset of each batch uses Data-Forcing Distillation, where the teacher sees noised ground truth. Inference is four steps; guidance is baked in, so there is no unconditional second pass.
Serving: eight RTX 5090s, one stream per GPU, 832×480 at a client-visible 24 FPS, 24.63 unpaced. Visual KV uses a fixed sink plus a sliding window; after a prompt switch the first new frame is pinned, still inside a fixed budget. Recurrent blocks decode with lightweight TAEHV. H.264/fMP4 fragments go into a bounded live-edge ring.
WBench Navigation, 158 cases, official images and prescribed actions, generated at 1248×704 with four steps per block at 24 fps, scored on quality, setting, interaction, consistency, and physical plausibility.
| Model | Avg. | Consistency | Physical |
| JoyAI-Echo-1.5 bidirectional | 81.6 | 89.8 | 70.6 |
| JoyAI-Echo-1.5 4-step | 81.0 | 88.3 | 70.1 |
| Zing-0.5 5B 4-step | 81.0 | 88.5 | 73.8 |
| HiDream-O1-World | 80.9 | 88.0 | 73.3 |
| Alaya-EVOKE 3-step | 80.8 | 86.9 | 72.1 |
| LingBot-World v2 fast | 79.4 | 86.5 | 69.1 |
The 5B four-step model ties Echo-1.5 four-step on the average and leads this table on physical plausibility at 73.8. Those numbers measure navigation-conditioned generation. Joint control is qualitative: a sled rider cheers and opens an umbrella on a text cue without restarting, while keyboard navigation continues. Other sessions ignite a tree into golden sparks, melt a candy house, walk through a mirror into a garden, raise a rainbow over a city, drop fiery rain on a battlefield, lift a character into flight, and have a mount breathe fire. Estimated rental cost is about $0.009 per stream-minute. Weights, inference code, and the Zing-SGLang serving stack are to be released.
Playability is defined here as keyboard plus text in one session, not as a higher navigation-board score. Five billion parameters, four steps, one GPU per stream, under a cent a minute: that is a cost at which people will actually retry. Conditioning on magnitude-aware keys rather than camera trajectories keeps data collection and the train/serve interface aligned, and it avoids pose error compounding on long rollouts. A segment-level teacher supervising a block-level student is a clean split for few-step causal world models that still need event-scale signal.
On the navigation board this is a tie with larger or bidirectional systems, not a gap. Joint control still has no matching quantitative benchmark.
The authors are blunt: visible responses in selected intervals do not show that consequences survive a later viewpoint change or a later interaction. The model does not track entity state or enforce transition rules. Facts that should persist have to live in visual context, text, and directional actions. Leaving the frame is not the same as ceasing to exist; an event ending is not the same as its effects ending; a bounded KV cache can drop the evidence. Autoregressive errors become later context. Long-sequence training and history perturbation improve video continuation. They do not add a persistence mechanism. Prolonged DMD introduces high-frequency noise and weaker color saturation. WBench does not test online text. The $0.009 figure is a rental estimate, not a measured bill.