Masked diffusion language models boost controllable world models for agentic RL

PatronusAI · hf · 2026-07-22

Masked diffusion language models improve controllable text world models for agentic RL

This paper argues that agentic reinforcement learning needs richer and more diverse training environments than hand-curated tasks with fixed difficulty. It reframes text-based world modeling as a steerable transition-dynamics problem with explicit anchors such as initial state, task context, tool schemas, domain rules, and steering directives.

What they built

A dataset of 239,403 grounded state-action trajectories across 9 open-source environments and 12 frontier model families.
A comparison between autoregressive world models and masked diffusion language models (MDLMs).
A plug-and-play GRPO training framework with deterministic state checks.

Main findings

MDLMs outperform AR models on coherence, groundedness, and rollout diversity, even when the AR models are more than 4× larger, at similar inference latency.
In zero-shot transfer tests on ScienceWorld, ALFWorld, and AppWorld, the method delivers up to 47% absolute gains over baselines without environment-specific fine-tuning.
The authors also analyze adversarial failure modes and run human evaluations on realism, correctness, and training utility.

The paper is open-sourced and positioned as a step toward scalable, steerable world models for agent training.

Original post →

Masked diffusion language models boost controllable world models for agentic RL

More from Research