Simplified Matching Distillation: few-step diffusion without adversarial training, validated up to 33B

ArashVahdat · x · 2026-10-24

The team introduces Simplified Matching Distillation (SMD) for few-step diffusion: no auxiliary score model, no adversarial training, no generator modification.
Core idea: the student learns mean flow by matching the teacher with the sum of a straight-line prediction and a curved zero-integral path at random sub-intervals; at sampling you just drop the curved part.
Highlights: works across images, video, joint audio–video and 3D, validated at scales up to 33B parameters; supports multi-teacher capability merging (like OPD in LLMs), cross-model transfer, and bidirectional teacher-to-causal distillation; photorealistic output, memory-efficient training, diverse mode-covering results.
Paper, code, model and data fully released. Retranslator sedielem calls it 'very clever, but not that simple IMO'.

More from Multimodal