Physis-Lang: Self-Evolving Language as a Physical Representation for Video World Model
Liming Lu, Xianzheng Ma, Wenkun He, Guanqi Zhan, Yilin Zhao, Junyu Chen, Mengyao Xu, Jiaojiao Fan, Wenhang Ge, Yuchao Gu, Yunze Liu, Boyi Li, Zhen Dong, Victor Prisacariu, Ming-Yu Liu, Song Han, Han Cai
cs.CV
2026-10-01
Self-evolving physics captions plus deficiency-driven data retrieval give Cosmos3-Nano a 6.22-point average gain across four physics benchmarks, beating Veo 3.1 on three.
Video generation models routinely render frames that look right and behave wrong: implausible collision outcomes, objects that vanish, fluids that slosh unnaturally. Multiple physics benchmarks have shown that perceptual realism does not imply physical understanding. The dominant fix assumes natural language cannot carry physics, so researchers bolt on visual signals, latent physics branches, numerical simulation, or planning modules, and leave language to describe what the scene should contain.
A team from NVIDIA, MIT, and Oxford asks the opposite question: maybe language was simply underused. If a caption spells out the full causal chain of butter melting, the heat supply, the phase change, the slump under gravity, can plain text carry physical knowledge? Their answer is yes, and it carries an open-source model past a proprietary flagship on physics evaluations.
Physis-Lang treats structured physical language as one shared, optimizable representation spanning data curation, training, and inference.
The caption format adds a physicsreasoning field on top of the base description, explicitly stating entities, causes, governing principles, temporal evolution, and effects. At inference time the model also receives a physicsnegativeprompt describing evolutions that should not happen in this scene (say, a stone floating motionless on water), used as negative conditioning to suppress common failures.
Caption quality comes from an agentic self-evolution loop:
PhysCapBench is a new benchmark built for this purpose: 246 physics-rich videos with 3,794 human-verified atomic assertions (15.4 per video on average), each tagged as Cause, Law, or Effect. Across nine iterations the loop lifts caption F1 from 78.64 to 87.82.
Data curation runs on language as well. A diagnosis agent sorts model failures into physics categories (rigid-body motion, collision, fluid dynamics), aggregates them into a deficiency profile, and retrieves gallery videos by physics-tag matching rather than visual similarity, so added clips target weak domains while staying visually diverse. The final training set is 183K videos: 71K filtered from WISA-80K plus 112K retrieved. Training changes no architecture, only LoRA adapters on attention qkvo projections, applied to Wan2.1-14B and the full Cosmos3 family (4B/16B/64B).
Because re-captioning 183K videos through commercial APIs is expensive, the team distills two Qwen3-VL-4B models: PhysThinker-C for captioning and PhysThinker-U for inference-time prompt upsampling. Estimated external API cost drops from about $24.12K to zero for the fully local pipeline.
Main comparison, fine-tuned Cosmos3-Nano, all scores rescaled to a 100-point scale:
| Benchmark | Cosmos3-Nano | +Physis-Lang | Veo 3.1 | HunyuanVideo-1.5 |
| PhyGenBench | 61.67 | 71.04 | 65.63 | 48.33 |
| Physics-IQ Verified | 40.23 | 43.41 | 34.99 | 31.60 |
| PhyGround | 65.18 | 69.90 | 69.24 | 66.28 |
| VideoPhy-2 all / hard | 60.41 / 48.31 | 68.02 / 62.36 | 68.87 / 58.43 | 62.94 / 51.69 |
Three of four benchmarks beat Veo 3.1; on VideoPhy-2 the full set trails by 0.85 while the hard split wins by 3.93. Gains transfer across backbones: Wan2.1-14B averages +7.05, and the Cosmos3 family gains +3.24 (4B), +6.22 (16B), and +5.02 (64B).
Ablations attribute the gains:
Three takeaways are directly reusable. Inference-time negative physics prompting is a training-free trick worth several points on any text-conditioned video model. Diagnosing failures by physics category and retrieving data by tag targets weaknesses more precisely than visual-similarity retrieval, a practical recipe for teams with tight data budgets. And PhysCapBench itself is a reusable asset: 3,794 human-verified assertions grade caption quality far more finely than holistic scoring.
To be clear about what this is: prompt expansion, negative prompting, and category-based data selection each have precedents. The novelty is wiring them into one self-evolving loop and systematically measuring the ceiling of language as a physics representation. Incremental in parts, but the combination is validated with substantial numbers.
The full VideoPhy-2 set still trails Veo 3.1. The fully local pipeline pays a price: replacing GPT models with PhysThinker cuts the average gain from 7.05 to 4.76, and Cosmos3-Edge-4B gains only 3.24, so smaller models benefit least.
There is an evaluation wrinkle the paper does not ablate: the original offline evaluators of VideoPhy-2 and PhyGenBench were replaced with GPT-5.5 for being more discriminative, yet the training captions are also GPT-5.5-generated. Scorer and data source share one origin, which invites systematic preference. All four benchmarks rely on LLM/VLM judges and no human cross-check is reported, so measured physical correctness is capped by judge quality.
On the engineering side, physics prompts run long. Wan2.1's UMT5XXL text encoder caps context at 512 tokens, forcing a sliding-window embedding scheme with 192-token overlaps, a sign of real pressure on text-conditioning capacity. The evolution loop and critic also depend on closed proprietary models (GPT-5.5, Gemini-3.1-Pro), so the representation's ceiling is set by their physics understanding.