eDMD fits the fake model to final images only, topping 4-step HPSv3++ on four backbones

eDMD: Endpoint Distribution Matching for Few-Step Diffusion Distillation

Tong Zhao, Xianfang Zeng, Yucheng Han, Ruoyu Wang, Mingkun Lei, Gang Yu, Chi Zhang

cs.CV

2026-11-13

eDMD fits the fake model to final images only and corrects every step; 4-step HPSv3++ leads on four backbones, FLUX.2-klein-4B at 12.70 versus DMD2 at 11.34.

What problem this solves

Few-step distillation compresses a diffusion or flow-matching teacher that needs tens of network evaluations into a student that samples in four. Inference returns only the last image.

Distribution matching distillation (DMD) trains a fake score model online on the student's samples and updates the student by the gap between the teacher denoiser and the fake denoiser. In one step, that prediction is the delivered image, so the two distributions are the same object. In several steps they are not. DMD2 trains one fake model on clean predictions pooled across steps, which fits a mixture. TDM builds a reference for each stage's intermediate state and matches the teacher inside that noise interval. The objective is tied to intermediate distributions. The image that counts is the endpoint.

A few-step student does not need its intermediate states to copy the teacher's marginal at the same noise level. After the final images already match, an intermediate reference can still leave a residual and keep pushing. Under exact denoisers, that residual is identically zero if and only if the fake model represents the endpoint distribution. On a 2D tree-shaped toy, an endpoint reference issues no update once final samples match the teacher, while mixture and stage references still move them. Two students with the same final samples and opposite intermediate bias receive residual updates with median cosine -0.97. Trained on from a matched student, the stage-marginal, importance-weighted, and mixture references drive energy distance to 0.18, 0.092, and 0.19. The endpoint reference stays at 0.004.

Method

eDMD trains the fake model only on the student's final images and uses that single model to correct every denoising step. Student and fake model are both initialized from the teacher. Training uses prompts only, with no real images.

Guidance uses each teacher's default scale: 7 on SD3-Medium, 4 on both FLUX.2-klein models, and 1 on Qwen-Image-2.1. The Qwen run also strengthens the teacher with perturbed-attention guidance. SD3 and the two FLUX models train for 2,000 updates on 191,500 prompts. Qwen trains for 600 updates on rewritten prompts.

Results

The image suite is DrawBench, 200 prompts and five seeds, 1,000 images. DPG is measured on DPG-Bench instead. The reported checkpoint is the best HPSv3++ over raw and EMA weights, checked every 100 updates. DMD2 is trained without the GAN loss from its original recipe. Teacher samplers and official few-step checkpoints are listed as unscored references.

BackboneeDMD HPSv3++Best matching baselineUnranked reference
SD3-Medium11.171D-DMD 10.84128-step teacher 8.554
FLUX.2-klein-4B12.699DMD2 11.337official 4-step 10.509
FLUX.2-klein-9B12.431D-DMD 11.325official 4-step 11.526
Qwen-Image-2.114.691DMD2 14.2128-step Turbo 13.850

eDMD posts the highest peak HPSv3++ on all four backbones, and the highest raw HPSv3++ at the reported fixed budgets. On SD3 that is 11.171 at 1,000 updates and 10.820 at 2,000, still above D-DMD's 10.677 at 2,000. On FLUX 4B it is 12.072 at 1,000 and 12.699 at 2,000. Among ranked students, eDMD also leads DPG and HPSv2.1. On Qwen it leads DMD2 and TDM on all seven reported scores.

It does not lead every column. Official CDM on SD3 scores HPSv3 at 11.890 against eDMD's 11.336, and PickScore at 22.622 against 22.589. The official 4-step FLUX 4B checkpoint scores PickScore 22.815 and DPG 85.79, against eDMD's 22.721 and 83.41. CLIP usually remains higher for the teacher or the official checkpoint. Four-step students pass the many-step teachers on preference models. The alignment score does not move with them.

Dropping the endpoint reference hurts most: SD3 falls from 11.171 to 10.122, and FLUX 4B from 12.699 to 11.095. Dropping PT leaves 11.026 and 11.667. Dropping query calibration leaves 10.798 and 12.507. Reruns of the same setup have standard deviation about 0.074 on SD3 and 0.11 on FLUX 4B. The endpoint gap is larger than that noise. On FLUX the calibrated density is nearly uniform, and the query-density gap sits at the same scale as the noise. At a shared prompt and seed, FLUX 4B keeps more fur and grass.

Why it matters

The image a user sees is the last one, and the matching term now accounts for that image alone. Intermediate predictions can leave the teacher's marginal. On FLUX 4B their norm reaches 2.9 times the final image. Switching a trained student back to the mixture reference pulls those predictions toward the final image, and the final image gets worse.

PT and query calibration add teacher evaluations only while training. Under uniform step sampling, the short trajectory covers about 12.4% of queries on SD3 and 19.3% on FLUX. Student inference stays at four forwards. Keeping TDM's student objective and rollout, and changing only the fake model to the endpoint, improves both peak and fixed-budget scores on SD3 and both FLUX models. The comparison set is the DMD family. Consistency models and adversarial distillation are outside it.

Limitations

The if-and-only-if result assumes exact denoisers and a teacher target that is the forward noising of one clean distribution. CFG and the short PT trajectory generally are not that target. The reference residual still drops out, but a zero correction no longer means the student's sample distribution equals the teacher's. Four-step students often beat the default-step teacher on HPSv3++. The pattern fits a guided target pushing mass onto high-preference modes.

The main tables report no FID, precision, or recall. A higher preference score is not evidence that the student moved closer to the teacher distribution. DMD2 is run without its GAN, so the baseline is weaker than the published recipe. On SD3, TDM and CDM are official checkpoints; on FLUX, TDM is an in-house ℓ2 reimplementation. Training budgets are not fully aligned. On SD3, raw HPSv3++ is lower at 2,000 updates than at 1,000. The headline number is the peak along the curve.

PT collapses when it is applied to every query. On FLUX 4B, HPSv3++ falls to -1.23 at 1,000 updates. On SD3 it trails the full method by 1.43 at 500 updates. Query calibration is nearly uniform on FLUX 4B, so that term is not a shared gain across the four backbones. Every experiment is text-to-image.

Terms

Source

What people are saying

Related papers

All paper explainers