WorldFoundry: Towards Unified Infrastructure for World Intelligence
Juanxi Tian, Jiadong Hou, Baiang Li, Xin Lin, Zhengbo Jiao, Weijia Zhang, Yeying Jin, Truong Nguyen, Haitian Zheng, Zhifei Zhang, Jianfei Yang, Ming-Hsuan Yang, Felix Heide, Wei Xiong, Enze Xie, Wanhua Li
cs.CV, cs.AI
2026-11-07
Loop closure, not visual plausibility, defines a world model; the open-source WorldFoundry runs 270 models and 74 benchmarks across video, 3D/4D, interactive and embodied systems.
The stronger video generators get, the looser the label 'world model' becomes. A clip with smooth motion and plausible physics may still be synthesizing observations rather than simulating an environment. The OpenEnvision team (the author list includes Ming-Hsuan Yang and Felix Heide) draws a hard line: visual plausibility is neither necessary nor sufficient. The criterion is loop closure. An intervention must leave a trace in the state that constrains later predictions, and each new observation must revise that state. Systems meeting both conditions are generative world models; the rest are foundational generative models.
Definition is only half the problem. Video, 3D/4D, interactive world, and embodied policy systems each ship their own runtimes, input conventions, artifact formats, and evaluation protocols, so cross-model comparisons cannot separate model capability from pipeline noise. The phrase 'model support' blends four distinct claims: catalog inclusion, runnable integration, partial validation, and benchmark reproduction. The authors read the bottleneck as infrastructural as much as algorithmic, so they fix the definition first and build the tooling second.
The definition is three operators. State inference q estimates a latent state from the interaction history; a transition model p rolls that state forward under endogenous dynamics and external actions; an observation model g decodes outputs from the state. The loop closes through feedback: once a real observation arrives, the state must be re-inferred before further prediction. The formulation is functional, not architectural, so the latent state can be recurrent, object-centric, geometric, or token-based.
A capability progression sits on top. Perceptual grounding is the foundation and only interprets or synthesizes observations. A three-stage world-model spine builds on it: persistent state modeling, then dynamics-aware prediction, then closed-loop interaction. Controllable transformation is orthogonal; however good the editing, it constitutes none of the spine stages. Systems are characterized by capability profiles rather than assigned to a single level.
WorldFoundry itself has five components:
The pivotal design choice is decoupling execution from evaluation. Expensive generations become durable, contract-typed artifacts that can be rescored, compared, and audited without reloading the model, and a produced score no longer automatically counts as a valid benchmark result.
Experiments ran on 128 A100/H100 GPUs (80 GB). The goal is infrastructure validation, not a leaderboard: verify that catalog entries resolve into executable configurations, that native semantics survive across interfaces, and that heterogeneous outputs land as traceable evidence. Every model runner passed a documented end-to-end reference workflow.
| Item | Count / result |
| Model Zoo entries | 270 |
| Benchmarks | 74 |
| Metric implementations | 62 |
| OpenVLA on LIBERO-10, single-object task | completed in 179 steps |
| OpenVLA on LIBERO-10, multi-stage tasks | two tasks hit the 520-step limit unfinished |
The qualitative observations carry more information than the counts. In video generation, frame-level smoothness and event-level consistency are different things; a polished sequence can still violate contact relations. In interactive world models, recognizing a revisited scene is not the same as remembering changes the system itself caused, such as whether an opened door stays open. In 3D, completion of unobserved regions can look convincing while being geometrically wrong. Current benchmarks rarely test intervention persistence or feedback assimilation, which is exactly what the definition demands. The paper does not report how the 270 entries distribute across evidence tiers, and there is no cross-model quantitative comparison table.
For model selection and benchmarking work, this removes a lot of glue code: one interface across four system families, with outputs persisted uniformly and rescorable later. For researchers, the capability progression and the closure criterion supply a shared vocabulary and identify what evaluation should measure next.
It is an infrastructure paper. No new models, no SOTA claims; the contribution is engineering and measurement discipline. Against the closest prior system, OpenWorldLib, the claimed edge is evidence tiering and five-modality coverage, but the Table 1 comparison is a feature matrix without measured backing.
The authors' own list: inference and evaluation only, with general training infrastructure left to future work; attention backends, NVFP4 quantization, and multi-GPU parallelism are wired in but not systematically characterized; validation concentrated on A100/H100 with little access to B200/B300-class hardware; and entry maturity is uneven, with inclusion, runnability, and validation deliberately tracked as distinct states.
Two concerns from reading it. The empirical section leans on qualitative description and screenshots; the only step-level quantitative result is the OpenVLA LIBERO episode, and 'validated' means the runner executes end to end, not that a benchmark was reproduced. The 270 figure invites reading as 270 one-command models, and the paper gives no per-tier counts, so check the appendices before relying on an entry. The closure criterion is also binary while real systems mostly satisfy it partially; the paper itself notes that current systems sit unevenly on the progression, and the capability-profile view describes reality better than a single boundary.