UC Berkeley and NVIDIA Release AW-0, a Unified World Action Model for Robots

UC Berkeley, NVIDIA, and the University of Pennsylvania jointly released AW-0 (Agentic World Action Model), described as the first "world action model": a single generative model that does three things at once—generating explicit code for planning (skill invocations, object parameters, and control flow), generating future video to predict visual dynamics, and generating continuous motor control signals to drive the robot, with all three produced jointly via denoising so that semantic planning directly shapes future prediction and action.

Confirmed

• Architecturally, the model consists of three experts: Code (visually grounded skill programs), Video (future visual dynamics prediction), and Action (continuous robot motion control); the code stream provides continuous planning context within a single generation pass, eliminating the need for a separate autoregressive code-generation stage.
• Efficiency: on a single RTX 5090, AW-0's joint inference takes 230ms, 2.4x faster than autoregressive code generation.
• Training recipe: an automated annotation pipeline expresses roughly 3000 hours of demonstrations as Code-as-Policies-style programs, distilling repeated behaviors into 50 reusable skill programs; once the skill library is fixed, whole-corpus annotation can scale up.
• Execution: in the default mode, generated programs serve as conditioning for learned action chunks; in hybrid mode, program slices that pass validity and progress checks can be executed directly through perception and control APIs, otherwise falling back to model action chunks. On the RoboTwin visual-randomization setting, hybrid execution raises the success rate from 20.6% to 29.2%.
• Evaluation: covering 50 RoboTwin 2.0 tasks and 8 real-world tasks, spanning the YAM parallel gripper and the Tianji Sharpa dexterous hand; 80.8% success rate in the clean RoboTwin setting and a real-robot manipulation score of 71.1, both far exceeding baselines.
• Ablation examples: given the instruction "make the left bowl end up containing only the mango," AW-0 with the code stream predicts interacting with the mango, while the WAM without the code stream grabs the apple, showing that code makes target objects and manipulations explicit; hand-object alignment similarly improves in tasks such as card insertion.

Why it matters

Current robot learning typically separates planning, prediction, and control into different modules; AW-0 unifies all three into one generative model, where the generated code serves both as the planning condition for learned control and as a directly executable interface for skill reuse. This provides an integrated plan–predict–execute foundation for manipulation agents, combining efficiency (single-GPU, near-real-time inference) with performance (significantly surpassing baselines in both simulation and on real robots).

2026-10-27 ~ 2026-10-27 · 9 related posts

• AW-0 debuts: one generative world action model plans in code, predicts, and controls robots — trevordarrell · 2026-10-27
• [source] AW-0 unveiled: first world action model that plans in code, predicts video, controls robots — trevordarrell · 2026-10-27
• Inside AW-0: code, video and action experts jointly denoise in one Mixture-of-Transformers — trevordarrell · 2026-10-27
• AW-0 runs at 230ms on a single RTX 5090, 2.4x faster than autoregressive code planning — trevordarrell · 2026-10-27
• AW-0 pretrained on 3,000 hours of demos, distilling 50 reusable skill programs — trevordarrell · 2026-10-27
• AW-0's code stream fixes world model prediction: picks mango where baseline grabs apple — trevordarrell · 2026-10-27
• AW-0 hybrid mode lifts RoboTwin success from 20.6% to 29.2% under visual randomization — trevordarrell · 2026-10-27
• [source] Berkeley and NVIDIA's AW-0 hits 80.8% on RoboTwin and 71.1 rubric score on real robots — trevordarrell · 2026-10-27
• [source] AW-0 hits 80.8% on RoboTwin tasks, beating strongest baseline by 16 points — trevordarrell · 2026-10-27