Length-Adaptive Decoding for Masked Diffusion Machine Translation
Yan Zhan, Mengkai Hou, Wanting Zhang, Zhijun Gao
EMNLP 2026
cs.CL, cs.AI, cs.LG
2026-08-23
Masked diffusion MT must pick a target canvas before denoising. Entropy-Valley scores a few all-mask lengths by mean predictive entropy, with no extra training. It recovers 64.9% and 65.3% of the oracle-length COMET gain on En→Zh and Zh→En versus a corpus-ratio baseline.
An autoregressive translator stops at EOS. A masked diffusion LM receives a fixed-length target canvas and unmasks it. Training puts EOS after that canvas, so at test time the model fills whatever length it is given. A short canvas drops verbs, placeholders, and numbers. A long canvas leaves room for repetition.
Most diffusion decoding papers study unmasking order. Length is either the reference translation, which is an oracle, or one corpus-level ratio, which cannot serve short commands and entity-heavy sentences alike. Classic NAT length heads need extra training, which does not fit a decoder-only diffusion backbone after LoRA-SFT.
Entropy-Valley asks the frozen model which canvas it is ready to fill. From a source sentence it builds at most five candidate lengths from a fixed ratio grid, including an EOS slot. Each candidate gets one all-mask forward pass. Mean predictive entropy over non-EOS slots ranks the canvases. The lowest-entropy valley is then decoded with the same 32-step minimum-entropy schedule as the baseline.
The grids sit around WMT19 medians: En→Zh {0.70…0.90}, Zh→En {1.00…1.40}, En→De {1.50…1.90}. Inference never uses the current reference length and adds no length predictor. Extra cost is at most five probe forwards on top of 32 denoising steps.
A too-short canvas packs too much source into too few slots. A too-long canvas creates positions with no confident token. Mean entropy sees that mismatch before the full denoising loop.
LLaDA-8B-Base is LoRA-SFT on 200k WMT19 pairs per direction, three seeds, 2,037 WMT22 sentences, 32-step MED.
| Direction | Fixed ratio | Entropy-Valley | Oracle length | Gap closed |
| En→Zh COMET-22 | 0.8345 | 0.8517 | 0.8610 | 64.9% |
| Zh→En | 0.8266 | 0.8431 | 0.8519 | 65.3% |
| En→De | 0.7170 | 0.7240 | 0.7382 | 33.0% |
sacreBLEU rises by 1.85, 1.63, and 0.82. Paired tests are solid on En↔Zh and weaker on En→De. Giving the ratio baseline the same or more forwards recovers at most 0.001 COMET.
A matched LLaMA-3-8B AR baseline is tied with EV on En→Zh (85.07 vs 85.17 on the 0–100 scale) and behind on Zh→En (83.68 vs 84.31). On En→De the AR model remains about 8.8 points above EV and about 7.4 above even the LLaDA oracle, so that gap is not a canvas bug.
With reference lengths fixed, tested unmasking orders span 0.0081 COMET on En→Zh. With MED fixed, length sources span 0.0265. EV stays far from oracle length error while recovering about 65% of the COMET gap: it picks a fillable canvas, not the human length. Placeholder retention on En→Zh moves from 66.9% to 89.1%, numbers from 77.6% to 81.3%. Three bilingual experts on 100 sentences per direction give Zh→En adequacy +0.50 and majority preference 45 vs 25; En→Zh adequacy +0.18 with mostly ties. Fluency barely moves.
The same protocol on Dream-Base and DiffuLLaMA still beats the ratio, with gap closure that varies by tokenizer and family.
For masked-diffusion MT, spend budget on length before inventing a new reveal order. In this setting the canvas moves COMET more. EV is cheap: five forwards and no new parameters.
More broadly, a fixed canvas removes stopping from generation. That decision has to be put back before denoising.
The ratio grid is direction-specific and was inspected on WMT22 diagnostic subsets, so it is not a fully hyperparameter-free method. It cannot choose a canvas below 0.70× source length on En→Zh. At T≥64 on En→De the fixed ratio slightly beats EV. Human eval is En↔Zh only. Absolute scores and gap closure change with the backbone; DiffuLLaMA closes less on Chinese, consistent with tokenizer coverage.