Decoding alone moves diffusion retrieval Hit@1 by 6.6-13.7 points, not the paradigm

_reachsumit · x · 2026-10-07

This arXiv paper challenges the simple narrative that diffusion beats or loses to autoregressive models in generative retrieval: recent work swapping autoregressive decoders for diffusion also changed identifiers, training recipes, and decoding at once, so reported gaps cannot be attributed to the paradigm.

Fixing identifier length and training budget, the authors train autoregressive, masked-diffusion, and block-diffusion retrievers on NQ320K and MS300K with residual-quantised, product-quantised, and random identifiers, then decode each model in multiple ways. Key findings:

Decoding alone shifts a diffusion model's Hit@1 by 6.6–13.7 points, meaning much of the reported gap comes from decoding, not paradigm.
A reference decoding scheme, generate-and-match, generates an identifier then retrieves the nearest corpus identifiers; the generated identifier is correct for 14–21% of NQ320K queries.
One-pass scoring — reading a fully masked identifier once and scoring each document by its codes' probabilities — matches or beats generate-and-match in 11 of 12 settings, and removes 46–83% of masked diffusion's deficit to beam search when starting from one sampled identifier.
Autoregressive models still lead in Hit@1; on NQ320K the lead comes from the model, not beam search.
On NQ320K, every paradigm largely memorizes which identifier answers which query: random identifiers retain 83–90% of the Hit@1 of residual-quantised ones.

Original post →

Decoding alone moves diffusion retrieval Hit@1 by 6.6-13.7 points, not the paradigm

More from Research