DFlash 2 Introduces Block-Diffusion Speculative Decoding to Speed Up GLM-5.3-Flash
songhan_mit · x · 2026-08-28
Inco AI released the DFlash 2 draft model, utilizing block-diffusion technology for speculative decoding to accelerate the inference speed of zai-org/GLM-5.3-Flash.
- Technical Mechanism: DFlash 2 is not a standalone language model; it operates within a speculative decoding server. It predicts an entire block of tokens in a single pass and employs two-tap dynamic convolutions to maintain draft quality, ensuring lossless decoding (greedy output matches the target model exactly).
- Deployment: It requires SGLang for serving, configured by setting speculative-algorithm to DFLASH.
- Performance: Benchmarks on GB300 GPUs indicate significant throughput improvements while adhering to the model's recommended sampling parameters (temperature 1.0, top-p 0.95), especially under high concurrency.
Related event: Inco AI Releases DFlash 2 Draft Model to Speed Up GLM-5.3-Flash Inference(2 posts)→
More from Infra
- Debunking AI data center myths: Water usage and grid impact — robleclerc · 2026-08-28
- Dev runs 125B Qwen model locally on M3 Max at 70 tok/s via MLX — mayfer · 2026-08-28
- NVIDIA launches Mesh open compute network to aggregate idle GPUs for AI — nvidia · 2026-08-28
- AI automated research finds numerical bug in vLLM/SGLang backend — josh_tobin_ · 2026-08-28
- Hot Chips 2026: inference chips enter an "era of ferment" with divergent bets — BenBajarin · 2026-08-28
- Testing Muon Optimizer: Smoother Gradients and Stable Residual Maxima — stochasticchasm · 2026-08-28