Frequency-aware remasking boosts diffusion LLMs by up to 7.33 points on MATH-500
ucriverside · hf · 2026-11-12
- Researchers from UC Riverside and Indiana University formalize test-time scaling for diffusion language models (DLMs) as adaptive allocation of both inference compute and decoding strategy.
- Method: A two-stage approach — a fast base sampler solves easy questions, then frequency-aware remasking targets hard ones; it tracks how often each token's prediction changes across denoising steps and preferentially commits frequently-changing tokens, encouraging exploration of alternative reasoning paths rather than collapsing onto the model's preferred trajectory.
- Results: On LLaDA-8B-Instruct and Dream-7B-Instruct, absolute accuracy gains of up to 3.73 (LLaDA) and 7.33 (Dream) points on MATH-500, and coverage gains of up to 11.39 and 10.98 points on HumanEval; on questions unsolved after 100 samples by the base strategy, it recovers solutions for 3.75%–18.75% of examples.
More from Research
- 50+ NeurIPS 2025 papers found containing AI hallucinations that slipped past peer review — CSProfKGD · 2026-11-12
- Indie researcher pauses RNN training over environment bugs, shifts to FunSearch-style tasks — cephaloform · 2026-11-12
- Self-supervised visual similarity partially explains word generalization in humans and machines — Patricio M. Vera · 2026-11-12
- P2Echo: one text-prompted model segments multi-view echocardiography with high accuracy and low cost — UBC-V · 2026-11-12
- I Fixed Alamo Drafthouse's Search With Query Logs Alone: +18.4% MRR, No Model Training — CShorten30 · 2026-11-12
- NYU lab open-sources Outerloop: autonomous research agents that open PRs only when benchmarks improve — mengyer · 2026-11-12