Blurring the Block Boundaries: Overlapping Block Denoising for Efficient Diffusion Language Models
Amir Mann, Jonathan Fhima, Shaked Brody, Or Avitan, Asif Zamir, Shahar Tsiper, Niv Nayman
cs.CL, cs.AI
2026-11-11
B3 denoises two adjacent blocks of a diffusion LM in shared forward passes with a self-calibrated spawn gate. Training-free: 11.5-14.5% speedup on three SOTA DLMs at on-par accuracy.
Diffusion language models (DLMs) treat generation as iterative denoising and predict many tokens per forward pass, which gives them a latency edge over autoregressive models. The dominant recipe is block diffusion: the output is split into fixed-length blocks, each denoised with bidirectional attention while attending causally to previous blocks through an exact KV cache. State-of-the-art models in this family also self-edit, refining a block over multiple passes until its slowest token is accepted.
The waste sits at the end of each block. Late in denoising, a pass accepts only a handful of tokens even though its compute cost is fixed, and the next block's easy tokens wait behind them. On DiffusionGemma 26B (HumanEval+), the low-acceptance regions near block boundaries consume 40% of the steps while producing 17% of the tokens; the last 20% of a block takes 22% of the steps for 10% of the tokens. Purely structural waste, independent of model capability, and consistent across models and datasets per the paper's appendix.
B3 blurs the boundary: once the current block is settled enough, the next block is initialised early and both are denoised in one shared forward pass until the current block is finalised. Three design pieces carry the method.
Training-free, hyperparameter-free, a sampler-level change.
Three models (DiffusionGemma 26B, LLaDA 2.1 mini 16B and flash 100B) on 10 benchmarks spanning math, code, reasoning and instruction following:
| Model | TPS | TPF | Avg accuracy |
| DiffusionGemma 26B | +11.5% | +19.4% | −0.40 pp |
| LLaDA 2.1 flash 100B | +13.6% | +15.9% | +0.76 pp |
| LLaDA 2.1 mini 16B | +14.5% | +16.3% | −0.28 pp |
TPS and TPF improve on all 30 model-benchmark pairs; accuracy averaged over the three models moves +0.03 pp, with no per-model or per-task tuning anywhere.
Two controls are informative. Enlarging the block, the obvious cheap alternative, does not recover the speedup: DiffusionGemma with B3 at block size 256 is 12% faster, a speed the baseline only matches at block size 384 or larger, at a cost of at least 4.0 pp accuracy where B3 loses 0.4 pp. Against the concurrent overlapping-block methods FlowBlock and MBD-LMs (both tuned per dataset) on LLaDA 2.1 mini, B3 is the fastest at block sizes 32 and 64, and 1.7 to 2.1 pp more accurate at 32. Neither baseline runs on DiffusionGemma-style USDMs; B3 does.
Speed is the main reason to deploy a DLM today, and B3 buys 11.5–14.5% throughput with no training and no measurable quality cost, pushing the latency-quality Pareto frontier. If you serve LLaDA 2.1 or DiffusionGemma, this is close to a free lunch: swap the sampler. The measurement that token acceptance collapses near block boundaries also stands on its own; future block-diffusion inference work has to reckon with this waste.