ByteDance's DScout boosts diffusion-drafted speculative decoding with 3.84x-4.14x speedups
ByteDance · hf · 2026-10-20
ByteDance, Virginia Tech and CMU introduce DScout, jointly optimizing training and inference for diffusion-drafted speculative decoding. The key insight: higher mean accepted tokens (MAT) doesn't guarantee higher throughput, since verification only accepts a consecutive prefix and per-round latency matters. DScout uses factorization-guided training (position-mixed training, intra-block unmasking, acceptance-weighted loss) and factorizes throughput as MAT over latency to guide target-conditioning depth, drafter depth and block size. Across six benchmarks it achieves 3.84x and 4.14x mean speedups over autoregressive decoding on Qwen3-4B and Qwen3-8B.
More from Infra
- Diamond-embedded copper vias shown to boost chip thermal conductivity — jwt0625 · 2026-10-20
- Stealth startup Physis debuts world foundation model play with PhysisServe, core tech headed to vLLM-Omni — vllm_project · 2026-10-20
- Barcelona Supercomputing Center AI Factory Accelerator Names Three AI Champions — NVIDIA Blog · 2026-10-20
- Luma's MARBLE: exact sequence parallelism for packed multimodal masks, wins 47/48 vs FlashMask+Ulysses — mrjonfinger · 2026-10-20
- Dual R9700 layer-split side effect: per-card load under 50%, fans stay quiet — yahbluez · 2026-10-20
- PyTorch Conference poster to break down KV cache offloading: when CPU beats shared storage — PyTorch · 2026-10-20