FlashMap: Flow Map Speculative Decoding
Jeff Shen, Zhejian Zhou, Ryan Lee, Jonathan May, Shirley Ho
cs.CL
2026-10-17
FlashMap drafts a full block in one flow-map pass at about half DFlash's size. On Qwen3-8B it wins every speedup: 7.4% over a matched DFlash on average, and 6.14× on GSM8K.
Speculative decoding asks a small model for a block of future tokens and checks that block in one forward pass of the target. Rejection sampling keeps the output distribution identical to the target, so the speedup does not change what the model would have said. At temperature 0 this reduces to the longest prefix that matches the target argmax. Acceptance length γ is the tokens committed per round, including the one token the target fills in at the first rejection. Per-token latency is about (draft time + verify time) / γ. Verification is memory-bound, and its cost barely grows with block length.
EAGLE-3 reads the target hidden states and accepts well, but it drafts autoregressively, so draft cost scales with the block. DFlash denoises a fully masked block in one pass. Draft cost no longer depends on block length, and the paper reports lossless speedups above 6×. That one pass scores each position on its own. The proposal is the product of per-slot marginals. The KL between the real block distribution and that product, the total correlation, stays even if the drafter gets larger. When the next words are new york or san diego with equal probability, an independent drafter emits new diego or san york half the time. Extra denoising steps recover the pairing and spend the latency a one-pass drafter is built to avoid.
FlashMap replaces that factorized proposal with a continuous flow on one-hot token blocks. The state moves from Gaussian noise at t=0 to the one-hot block at t=1. The only randomness is the initial noise, and all K slots share it, so one draw selects a mode for the whole block.
The backbone is a 6-block DiT, width 2048, 16 heads. Slots self-attend, cross-attend to target features, then go through an MLP. adaLN-Zero scales and shifts each of those by the flow time. The features are target layers 1, 9, 17, 25, and 33, projected at positions before the anchor, plus an embedding of the anchor token. The anchor is the last token the target has already committed. The embedding matrix and the LM head are copied from the target and frozen. Slot losses are cross-entropy, weighted by exp(-(k-1)/7), so early positions dominate: an early error discards every later token. Qwen3 has a vocabulary of 151,936, and the clean token is readable in the state only near t=1. Training samples τ uniformly, where τ is the fraction of the initial decoding error already removed, then maps τ back to t.
Integrating the ODE costs on the order of 100 evaluations. One look at pure noise has an optimum equal to the per-slot marginals, and the joint structure is gone. That structure appears only as later steps fold this particular noise draw into one coherent block. A flow map replaces the integral from time s to time t with a single evaluation. Distillation uses the semigroup: a direct jump must land where a jump to the midpoint and a second jump would land. Zero-length jumps match the frozen teacher denoiser. Longer jumps are trained against the student's own split of the interval. Half the blocks are zero-length jumps, and 1/16 of all jumps are the full jump from 0 to 1. Drafting takes one jump or two.
The data are 500K samples from OpenPerfectBlend, with block length 16. The flow trains for about 39K steps at peak learning rate 6e-4. Distillation runs about 23K steps at peak 5e-5. EAGLE-3 and DFlash are retrained on the same data and the same schedule. On Qwen3-8B, DFlash is about 1B parameters and FlashMap about 0.5B. EAGLE-3 uses a tree of size 16 so every method shares the verification budget. Timings use SGLang at concurrency 1 on one H200 NVL, against autoregressive decoding with the target.
One-step FlashMap is the fastest drafter on every benchmark at both T=0 and T=1. Speedup over autoregressive decoding at T=0:
| Method | GSM8K | MATH | AIME25 | MBPP | HumanEval | LiveCodeBench | MT-Bench | Alpaca | Arena-Hard |
| EAGLE-3 | 3.20 | 3.09 | 2.82 | 2.57 | 2.87 | 2.90 | 1.80 | 1.78 | 1.92 |
| DFlash | 5.77 | 5.59 | 4.67 | 3.74 | 4.50 | 4.88 | 2.48 | 2.34 | 2.67 |
| FlashMap 1-step | 6.14 | 5.89 | 4.89 | 4.02 | 4.73 | 5.26 | 2.73 | 2.56 | 3.04 |
| FlashMap 2-step | 5.89 | 5.57 | 4.61 | 3.79 | 4.53 | 5.06 | 2.47 | 2.33 | 2.76 |
One step is 7.4% faster than the matched DFlash on average. The high is 6.14× on GSM8K. At T=0 the paper splits that gain: math acceptance and speedup are only 2.3% and 5.5% above DFlash, while chat is 7.4% and 11.1% above. At T=1, one step still leads all nine tasks. GSM8K drops to 5.12× and Arena-Hard to 2.30×. Two steps are slower than one step in all 18 speedup cells.
Acceptance length ranks the drafters differently. At T=0:
| Method | GSM8K | MATH | AIME25 | MBPP | HumanEval | LiveCodeBench | MT-Bench | Alpaca | Arena-Hard |
| EAGLE-3 | 6.75 | 6.43 | 5.96 | 5.38 | 6.04 | 6.16 | 3.75 | 3.71 | 3.97 |
| DFlash | 7.43 | 7.09 | 5.92 | 4.77 | 5.74 | 6.35 | 3.14 | 2.96 | 3.38 |
| FlashMap 1-step | 7.69 | 7.24 | 6.00 | 4.95 | 5.82 | 6.65 | 3.34 | 3.14 | 3.71 |
| FlashMap 2-step | 8.31 | 7.69 | 6.36 | 5.24 | 6.26 | 7.19 | 3.40 | 3.20 | 3.80 |
Two-step FlashMap beats DFlash on γ in all 18 settings, by 10% on average, and still wins 13 of the 18 speedups. One-step FlashMap, at about half the parameter count, has higher γ in 15 of 18 settings. On MT-Bench at T=0, EAGLE-3 reaches 3.75, above two-step FlashMap at 3.40. At T=1, EAGLE-3 has the best γ on 6 of 9 tasks, 9% longer on average, and is 35% slower end to end.
The shuffle test separates per-slot accuracy from agreement across slots. Shared noise versus an independent draw per slot: Δ is the acceptance that comes from joint structure. DFlash and a raw flow evaluated once both have Δ of 0. A 16-step flow reaches 0.82, 0.59, and 0.29 on math, code, and chat. One-step FlashMap reaches 0.32, 0.13, and 0.08. Two steps reach 0.78, 0.47, and 0.25, close to that 16-step teacher. Conditional acceptance tells the same story. One step decays faster than DFlash late in the block. Two steps stay flat, and slot 16 is higher than DFlash in every domain. Drafting several blocks from independent noise and verifying them as a tree can add about 11% over a single chain. That grid is not in the main tables.
The piece you swap is the drafter. The target sampling distribution stays put. On Qwen3-8B the fastest point is one forward pass at about 0.5B parameters, and it beats both DFlash and EAGLE-3 on every speedup. Math and code land around 4× to 6×. Chat lands around 2.5× to 3×. The large multiples are on structured text.
Against DFlash, same data and the same verification budget, 7.4% is a real increment and not a new regime. Against EAGLE-3, acceptance trades wins, and the end-to-end gap is draft latency. At one step, Δ from joint structure is only a few tenths of a token. If the serving stack can afford a single draft pass, most of the gain is a slightly better marginal plus a smaller network.
The targets are Qwen3-4B and Qwen3-8B only. Other families and larger models are untested, and the 4B numbers sit in the appendix. Every drafter, baselines included, trains on a 500K subset of OpenPerfectBlend, so absolute γ should not be compared with drafters trained on the full set. Speed is concurrency 1 on one GPU. Batched serving is not measured. The benchmarks are English math, code, and chat, at temperatures 0 and 1. Follow-ups that sit on a DFlash draft, including draft trees, autoregressive postprocessing, and low-rank latent mixtures, are not in the comparison. The runs may also be short: the flow-language-model work they build on trains for about 1M steps and distills for about 100K, against 39K and 23K here.
The fastest configuration does not fully capture dependencies inside the block. Conditional acceptance for one step falls faster than DFlash toward the end of the block, so a longer block will not help much. Under shared noise, both one step and two steps still trail the 16-step flow. Under shuffled noise the marginals are worse as well. Each point on the block-length curve is a single run, and the paper says so. The cost of training at K=16 and drafting shorter blocks is quoted from DFlash as about 0.15 tokens, not remeasured here.