Microsoft's SMD distillation hits 4-step generation that beats multi-step diffusion teachers
microsoft · hf · 2026-10-15
Microsoft Research, with NYU and HKUST, present Simplified Matching Distillation (SMD), showing simple on-policy velocity regression suffices for 4-NFE generation without auxiliary score models or adversarial training. A training-only residual proxy called Pivot separates local corrections from the student's interval-mean prediction, and students are repelled from weak teacher predictions. Evaluated across image, 3D, video, and joint audio-video generation up to 33B parameters, SMD at 4-NFE outperforms strong few-step baselines and even surpasses multi-step teachers on several benchmarks, supporting cross-model, multi-teacher, and causal-video distillation.
More from Multimodal
- Runway CEO reacts to claim it brought DeepMind's world model theory to life first — c_valenzuelab · 2026-10-15
- Minimax beats Seedance for character swap: 3 minutes, 4 steps, no extra LoRA — agapes1270 · 2026-10-15
- Flux 3 Klein wows with precision collage generation, coming soon — makeitrad1 · 2026-10-15
- anatomy Skill Hits 10.3k Stars: One Prompt Turns Ideas Into Interactive Explainers — vista8 · 2026-10-15
- AI-generated pod racing on Mario's Rainbow Road is the crossover demo of the day — draginol · 2026-10-15
- Musk amplifies Grok Imagine demo turning one image into a multi-shot video — elonmusk · 2026-10-15