Adobe/MIT: small critics silently break DMD distillation — joint scaling + STAC halve FID
rzhang88 · x · 2026-11-03
Adobe and MIT researchers show that in Distribution Matching Distillation, a bigger teacher does NOT help if your critic stays small — the student can get worse or even collapse.
Two simple fixes:
- Joint scaling: grow the critic alongside the teacher so students benefit from larger teachers
- STAC: a step-aware critic conditioned on the student's generation step for step-specific score estimates
Evidence:
- Controlled experiments across seven SiT sizes on ImageNet
- Joint scaling + STAC cut the 4-step SiT-B student's FID from 13.91 to 6.90 without CFG
- Validated on SDXL, FLUX.2, and Wan2.1 text-to-image/video (e.g., 4-step FLUX.2 4B student with a 9B teacher + 9B critic)
Full ablations on critic design and training strategies in the appendix.
More from Multimodal
- Testing HeyGen Video 1's creative limits with two distinct prompt styles — toolstelegraph · 2026-11-03
- HeyGen CEO: open models will win video and voice just like they did LLMs — toolstelegraph · 2026-11-03
- NVIDIA Cosmos Simulation Recipes Turn Daytime Drives Into Rainy Nights With Trajectories Intact — NVIDIAAI · 2026-11-03
- Suno user begs for a button that writes prompts so they never have to — gleech · 2026-11-03
- Seedance 2.5 demo: a 30-second swamp creature short film made in Dreamina — LudovicCreator · 2026-11-03
- Turning his photo library into drum samples with Gemini 4 — alexanderchen · 2026-11-03