DStill Distills Multi-Step Denoising Into One Pass, 6.45x Speedup for Diffusion Speculative Decoding
LucaAmb · x · 2026-11-02
arXiv paper "DStill: Trajectory Consistency Distillation for Accelerated One-Step Drafting in Block Diffusion Speculative Decoding" tackles a core limit of one-step drafting in discrete diffusion: token factorization error, where tokens in the same masked block are predicted independently, producing inconsistent drafts and shorter acceptance lengths.
Method (two stages):
- Stage I: train a target-conditioned drafter to complete partially revealed blocks, committing high-confidence tokens first and conditioning the rest on them
- Stage II: distill the drafter's own multi-step refinement trajectories into a one-pass student, using a curriculum that gradually shifts from partial to fully masked inputs
Results: across reasoning, math and code benchmarks on three model families, DStill improves acceptance length substantially and achieves up to 6.45× speedup over autoregressive decoding, while keeping drafting to a single forward pass at inference.
More from Research
- Adobe & UCLA's Tev: verifiable progress rewards lift long-horizon agent Pass@5 from 0.17 to 0.28 — jiqizhixin · 2026-11-02
- Old trick still works: portal sampling cuts path-tracing noise dramatically at 64spp — ssh4net · 2026-11-02
- Marcel Böhme's paper asks why cybersecurity can never be solved, proposes vulnerability-guided hardening — mboehme_ · 2026-11-02
- Perplexity open-sources WANDR: 500 research tasks, 170,495 evidence points, URL-rechecking grader — thisguyknowsai · 2026-11-02
- AI researcher: we have no reliable way to tell whether an AI system is conscious — dioscuri · 2026-11-02
- Ant Group's Venus-DefakerOne2 hits 96% accuracy detecting AI-generated videos — inclusionAI · 2026-11-02