New paper reframes score distillation as distribution matching, explains SDS mode collapse
burny_tech · x · 2026-09-23
An arXiv paper introduces Probability-Flow Distillation (PFD), reframing score distillation as distribution matching in parameter space.
- Unified theory: extends the particle variational inference view of VSD to SDS and SDI, showing SDS collapses onto target modes (explaining mode collapse) while SDI converges to a contracted distribution — and why SDI needs a negative CFG scale
- Method: noting the DDIM posterior mean equals one Euler step of the PF-ODE, the authors replace it with a full reverse solve, then drop a Jacobian to get PFD, which only requires solving the forward PF-ODE
- Results: experiments on synthetic targets, CelebA, and text-to-3D show VSD-level distribution matching with lower-variance gradients and faster convergence
More from Research
- Yale PhD student open-sources his paper figure scripts, packaged as a Skill for Claude Code and Cursor — burny_tech · 2026-09-23
- AI models now match superforecasters on ForecastBench; rematch set for October — burny_tech · 2026-09-23
- Dev uses Opus 5.5 with Lean to formally verify Claude Agent SDK, yielding 16 bug-fix PRs — bcherny · 2026-09-23
- Mathematicians, not just LLMs, made AI's math breakthroughs possible, scholars argue — tak3sh8 · 2026-09-23
- AI-enabled drug discovery cuts discovery time by 15-80%, McKinsey research finds — menhguin · 2026-09-23
- Gemini training details dissected: groupwise reward redistribution to fight reward hacking — nrehiew_ · 2026-09-23