Sakana AI says multiple diffusion language models improve test-time scaling on code and math

tkasasagi · x · 2026-07-22

Multiple masked diffusion models beat single-model test-time scaling

Sakana AI says its ICML 2026 paper, “UnMaskFork: Test-Time Scaling for Masked Diffusion via Deterministic Action Branching,” shows that multiple masked diffusion language models can collaborate on one answer to improve performance on coding and math tasks.

The paper studies test-time scaling: improving results by spending more compute at inference time instead of additional training.
Unlike standard LLMs, masked diffusion language models generate text by gradually unmasking tokens, which enables parallel generation and a different way to search for answers.
The authors found that the usual temperature-based diversity trick does not work well for Dream-Coder-style MDLMs.
Their method, UnMaskFork (UMF), creates diversity by switching among models rather than adding randomness.
Multiple MDLMs split the unmasking process, while Monte Carlo Tree Search explores promising assignment orders.
The approach keeps generation quality while exploring diverse solutions, and reportedly consistently outperforms existing test-time scaling methods on coding benchmarks, with steady gains on math as compute increases.

Original post →

Sakana AI says multiple diffusion language models improve test-time scaling on code and math

More from Models