SSAD 2026: Architecting and Interacting with Autonomous Driving World Models

At the SSAD 2026 autonomous driving world model workshop, @abursuc gave a series of talks systematically laying out key considerations for building autonomous driving world models, covering mainstream architectures, state representation design, and the challenges of achieving true interactivity.

Confirmed

• Most current world models share a common architecture: a tokenizer compresses and produces state representations, while a world model module consumes the current state and conditions to output the next state. He credited Wayve's Alex Kendall and team for articulating this idea to the community early.
• The key divergence across methods lies in what the tokenizer encodes and how it is trained: options include frozen self-supervised visual features, semantics-, geometry- or task-relevance/predictability-oriented designs, or distillation from DINO.
• State representations must faithfully reconstruct pixels so the model can run internally, while avoiding over-focusing on high-frequency detail.
• States must accommodate information streams at different frequencies—for example, world model states need action updates every 10Hz—making the carrying and updating of multi-frequency information a core challenge.
• Interventions on the environment may induce correlations between post-intervention states and actions, raising two key questions: how to enable and integrate interactivity in the state space, and how to represent the action space.

Why it matters

This synthesis unifies design choices scattered across papers under the axis of "what the tokenizer encodes and how it is trained," providing a clear framework for comparing and studying AD world models; multi-frequency state updates, state-space interactivity, and action space representation remain open problems on the path to truly interactive world models.

2026-09-17 ~ 2026-09-17 · 7 related posts

• [source] Anatomy of modern world models: a tokenizer compresses states, a module predicts the next — abursuc · 2026-09-17
• World models share one architecture — the tokenizer is where methods diverge — abursuc · 2026-09-17
• SSAD 2026 talk maps the zoo of state representation designs for world models — abursuc · 2026-09-17
• Building world models for autonomous driving: state representation trade-offs — abursuc · 2026-09-17
• [source] Key considerations for building autonomous driving world models — abursuc · 2026-09-17
• [source] Course thread: interventions get hard when states and actions are correlated — abursuc · 2026-09-17
• World model interactivity: handling mixed-frequency state updates at 10Hz — abursuc · 2026-09-17