NVIDIA releases post-training framework for real-time interactive Cosmos world models

chenhsuanlin · x · 2026-10-22

NVIDIA released a post-training framework for real-time interactive world simulation for Physical AI, built on Cosmos 3 world foundation models. Key points:

Achieves real-time interactivity via causal training, distillation, and optimized streaming inference
Post-trained models stream visual observations through autoregressive generation, responding to incoming actions and scene controls in real time
Ships example checkpoints for robotics, autonomous vehicles, and sim-to-real transfer
Includes optimized streaming inference code

The authors note that while Cosmos provides broad visual knowledge, each simulation setting has its own controls, data, and evaluation needs — the shared recipe and example models give researchers a practical starting point for building specialized interactive world simulators.

Original post →

NVIDIA releases post-training framework for real-time interactive Cosmos world models

More from Embodied