StepRefine: Step-Aware Decoding Cuts Hallucinations in Multimodal Diffusion LMs

Cornell0917 · hf · 2026-11-16

Researchers from Monash, Cornell, UC Santa Cruz, Northeastern, and Minnesota propose StepRefine, a plug-and-play decoding strategy that mitigates hallucinations in multimodal diffusion language models (MDLMs).

Key observations: mask uncertainty follows an overall downward trend across iterative denoising, but a few denoising steps show localized entropy spikes closely associated with hallucinations. The authors argue that committing to discrete tokens discards distributional information at masked positions, limiting continued interaction, and identify anomalous steps by measuring each step's entropy deviation from its neighboring temporal window.

StepRefine switches between discrete denoising and latent refinement based on entropy spikes and distributional stability, reducing hallucinations without retraining the model.

Original post →

More from Multimodal

Multimodal channel →