FlashMap: flow map speculative decoding delivers >6x faster Qwen3-8B inference
kalyan_kpl · x · 2026-10-20
FlashMap, a new speculative decoding algorithm from Princeton/USC/Polymathic AI, achieves >6x speedup on Qwen3-8B with exactly identical outputs. The drafter is a DiT that cross-attends to target hidden states and drafts a whole block of K tokens in one flow map evaluation, modeling joint inter-token dependencies — unlike slow autoregressive drafters or DFlash's independent parallel slots. The target verifies all K tokens in one pass. Drafters for Qwen3-8B/4B are on Hugging Face, usable via Transformers, vLLM, or SGLang (tested on 1x H100, greedy, concurrency 1).
More from Infra
- Cloudflare cut its AI code reviewer's inference bill by 68% with prompt caching and model routing — DanielLockyer · 2026-10-20
- DeepSeek launches DeepInfer with open prefill/decode benchmarks and reproducible scripts — Xianbao_QIAN · 2026-10-20
- Auto-research on idle GPUs: a CoreWeave pipeline that keeps agents busy — _ScottCondron · 2026-10-20
- Huawei's 960DT chips: most 2027 supply reportedly earmarked for DeepSeek supercluster — teortaxesTex · 2026-10-20
- €550 Used HP Z640 With Dual RTX 3060s: Reader Asks for MoE Local LLM Build Sanity Check — ElekDn · 2026-10-20
- NVIDIA GTC Berlin kicks off with Jensen Huang keynote on October 21 — IanAndrewsDC · 2026-10-20