ICML 2026 Paper SAS: Optimizing Thought Scheduling in Diffusion LMs

ICML 2026 will feature a paper on diffusion language model inference titled "Scheduling Thoughts." The research introduces Self-Aware Scheduling (SAS), suggesting that the next breakthrough in LLM reasoning lies in scheduling thoughts rather than simply generating longer chains of thought.

Background and Core Concept

Unlike autoregressive models constrained by left-to-right serial generation, diffusion language models start from masked text and iteratively reveal tokens, offering freedom in the sequence of thought emergence. The key to decoding is not just parallelization, but learning "when to commit." Existing heuristic decoding schedules are short-sighted, whereas SAS introduces a lightweight scheduler that uses the frozen model's own path likelihood as a reward to learn a native, easy-to-hard curriculum.

Key Details and Experimental Gains

SAS transforms decoding order into a trainable strategy akin to process supervision without human trajectories. On the Sudoku task, SAS achieved 91.8% accuracy (and 97.5% with a second fine-tuning phase), significantly outperforming the best heuristic at 82.0%. These gains transfer to other tasks: on LLaDA-8B, GSM8K pass@1 improved from 64% to 76%, and MBPP pass@1 increased from 39.5% to 41%.

Implications

A surprising finding is that human logical sequences are not always optimal for models. Pre-trained diffusion LMs might perform better along trajectories different from human problem-solving paths, pointing towards exploring "model-aware" thought orderings rather than just mimicking human CoT.

2026-07-06 ~ 2026-07-07 · 15 related posts

• ICML 2026: Self-Aware Scheduling (SAS) for Diffusion LMs — furongh · 2026-07-06
• Diffusion LMs Bring Freedom to the Order of Thought — furongh · 2026-07-06
• Diffusion LM Decoding Doesn't Have to Be Left-to-Right — furongh · 2026-07-06
• Key to Diffusion LMs Is Learning When to Commit to Tokens — furongh · 2026-07-06
• Existing Diffusion LM Decoding Schedules Are Too Short-Sighted — furongh · 2026-07-06
• SAS Turns Decoding Order Into Trainable Inference Strategy — furongh · 2026-07-06
• SAS Makes Schedulers Self-Aware — furongh · 2026-07-06
• SAS Acts Like Process Supervision Without Human Trajectories — furongh · 2026-07-06
• SAS Significantly Boosts Sudoku Solving Accuracy — furongh · 2026-07-06
• Human Logical Order Isn't Always Optimal for Model Reasoning — furongh · 2026-07-06
• SAS Gains Transferable Improvements on GSM8K and MBPP — furongh · 2026-07-06
• A New Dimension of Reasoning: When to Commit, Not What to Generate — furongh · 2026-07-06
• [source] Paper: Learning the Order of Thought in Diffusion Language Models — furongh · 2026-07-06
• ICML 2026 Poster: LLM Reasoning and Diffusion Models — furongh · 2026-07-06
• Scheduling Thoughts in Diffusion Language Models — furongh · 2026-07-07