DFlash2 delivers up to 3.03x decode-only speedup for Beam across five benchmarks
vllm_project · x · 2026-10-16
Reflection reports 1.72–3.03× decode-only speedups with DFlash2 across GSM8K, MATH500, HumanEval, MBPP and MT-Bench at concurrency 1, 16 and 32; GSM8K reaches 3.03×/2.51×/2.75× respectively.
Measurements use a disaggregated prefill/decode setup with prefill excluded. A separate tuning note: K=7 improved throughput even though the drafter was trained with block size 16.
More from Infra
- Meta's open Muse Spark needs 493GB of memory — far beyond any consumer laptop — TejasKumar_ · 2026-10-16
- CoreWeave plunges 20% as credit markets price >60% default odds; is this dot-com all over again? — kimmonismus · 2026-10-16
- MiniCPM-V 4.7 35B-A3B hits 17+ tokens/s on iPad with ~3GB memory — techNmak · 2026-10-16
- China rolls out 'token factories' and a 2026-28 token economy action plan — nordicinst · 2026-10-16
- Vercel Sandbox Drives go GA: attach TBs of data in sub-second, fork instantly — cramforce · 2026-10-16
- NVIDIA launches RTX Spark for local AI on Windows PCs — ostrisai · 2026-10-16