PhiZero: A World Model Built Around Physical Language
Shuyao Shang, Yuqi Wang, Ruopeng Gao, Xu Chen, Tieniu Tan, Lue Fan, Zhaoxiang Zhang
cs.CV
2026-07-31
PhiZero self-supervises a discrete physical language from in-the-wild videos to describe state transitions, reasons future evolution in that space, then renders video. It scores 41.2 on Physics-IQ Verified, ahead of Sora 2 and Cosmos3-Super.
Video world models generate high-fidelity footage and are increasingly treated as simulators of the physical world. But the mainstream approach predicts the future directly in pixel space, leaving physical dynamics implicit inside high-dimensional visual predictors, which often yields physically incoherent results: a ball hits a duck and the duck does not move, collisions have no consequences.
Humans do not memorize every frame. They abstract patterns of how the world evolves from visual experience, then organize and reason about them explicitly in language. Natural language works well in the digital world but is too coarse for the state transitions of the physical world. PhiZero asks: can we learn a physical language, finer-grained than natural language, for explicitly reasoning about world evolution?
PhiZero follows a reason-then-render paradigm, centered on a discrete physical language learned by self-supervision to describe state transitions in video. It has two parts.
The Physical Language Tokenizer compresses a video's state transitions into a discrete sequence. Rather than extracting a global representation of the whole video, it explicitly models the transition between each pair of adjacent latent states: a shared Q-Former draws a set of transition features from each adjacent pair, concatenated in time order, then discretized with Finite Scalar Quantization (FSQ). FSQ builds its vocabulary as the Cartesian product of scalar quantization levels and needs no separately learned codebook; here the vocabulary is 25K symbols, and a 33-frame video encodes into a 256-token physical-language sequence. A pretrained diffusion decoder (Wan2.2-5B) reconstructs the original video conditioned on the physical-language sequence plus the first frame. The first frame supplies static appearance, so the discrete bottleneck is forced to encode state changes rather than redundant static detail.
The Physical Language Reasoner is an autoregressive VLM (Qwen3-VL-4B) that, given the first frame and a textual action intent, predicts the corresponding physical-language sequence, which the trained diffusion decoder then renders into a future video. The whole factorizes into reasoning the physical language and rendering the video, separating dynamics inference from pixel synthesis.
For data, 50K hours of in-the-wild video are filtered down to about 10K hours for tokenizer pretraining, plus 1K hours of simulation video; the reasoner's SFT uses 5M clips filtered to 1M four-second segments.
Both generation and understanding are tested. On generation, three benchmarks: Physics-IQ Verified (physical outcome fidelity), PhyGround (physical-law adherence), and WorldModelBench (general world modeling). PhiZero ranks first on all three: IQ-Score 41.2, ahead of Cosmos3-Super at 39.5, Grok-Video at 34.8, and Sora 2 at 26.5; PhyGround Physics Score 3.01 and Overall 2.97, the highest; WorldModelBench Total 8.19, first.
| Benchmark | Strongest baseline | PhiZero |
| Physics-IQ Verified (IQ-Score) | 39.5 (Cosmos3-Super) | 41.2 |
| PhyGround (Overall) | 2.95 (Wan2.2-14B) | 2.97 |
| IntPhys2 (intuitive physics) | 55.63 (Gemini-2.5 Flash) | 56.34 |
The three understanding benchmarks are competitive too: IntPhys2 intuitive physics 56.34 beats Gemini-2.5 Flash at 55.63 and GPT-4o at 53.75; LikePhys physical-plausibility discrimination has the lowest average error at 41.7; YoCausal real-world causal understanding ranks first with an aggregate rank of 2.0. Understanding tasks use likelihood comparison: both videos in a pair are encoded into physical-language sequences, their log-likelihoods under the reasoner are computed, and the higher-likelihood one is selected as physically valid.
The tokenizer's own reconstruction quality is telling: 256 tokens at PSNR 28.9, higher than VideoFlexTok (288 tokens, 25.2) and far more compact than the Wan2.2 VAE's 44,800 continuous tokens. PhiZero also demonstrates action-conditioned driving and robotics world models, plus zero-shot motion transfer: human body motion to a G1 humanoid and hand motion to a Sharpa dexterous hand.
For anyone building world models and Physical AI, PhiZero offers a clear alternative to pure pixel prediction: move dynamics inference out of pixel space and into a learned discrete physical language. The physical language is both an intermediate for generating worlds and, in reverse, a tool for video understanding (judging whether physics is plausible); one representation serves both ends, which a pure pixel model cannot do.
Its transferability opens another thread. The state-transition patterns are abstracted from large-scale human video, and the authors note these patterns may transfer to robotic embodiments, easing the scarcity of real-robot interaction data. That is attractive for anyone who wants to train robots on human video.
The authors flag three. First, the physical language is an empirical representation of state transitions, not a symbolic formulation of physical laws; the discrete symbols are not yet grounded in interpretable physical variables or formal laws. Second, learned from observational video, its coverage is bounded by what is visually observable in the training data, so tactile interactions and microscopic particle dynamics are hard to model. Third, limited by data and compute, the current models are small and the training corpus remains limited relative to the diversity of the physical world.
The deeper concern is that the physical language is data-driven, so how close its learned physics is to real physics, rather than merely looking physical, is shown only indirectly through benchmark scores, with no interpretable analysis of which physical quantities (mass, momentum, friction?) the representation encodes. Zero-shot transfer to robots is an appealing prospect but currently only a qualitative demo, with no quantitative validation of transfer effectiveness on real robot tasks. The fixed-duration-clip limitation also remains; long-horizon world modeling needs hierarchical or recurrent prediction, which the authors list as future work.