ProVe flips speculative decoding: large proposer, small verifier yields 2.5-2.7x speedup
Ziqi Jin · hf · 2026-11-03
The paper proposes ProVe (Proposer-Verifier Decoding), reversing speculative decoding roles: starting from a standard autoregressive checkpoint, the large model is fine-tuned to predict a block of future tokens in one forward pass, then verify the block with one additional pass. The verifier can be a smaller AR model or the proposer itself (ProVe-lossless, reproducing greedy AR decoding).
Key results:
- With block size 16, the proposed block achieves an average AR-matching prefix of 6.6 tokens on GSM8K
- 32B proposer + ProVe-lossless: 2.5× average AR throughput across six benchmarks, without training a separate draft head
- With a 3B verifier: 2.7× average throughput at only 2.6 accuracy points below AR, with a controllable speed-accuracy tradeoff
More from Infra
- Darkbloom passes 1T tokens as Mac compute network pivots to CI/CD — gajesh · 2026-11-03
- OpenRouter Hits 100T Weekly Tokens; Surplus Jumps to 2.87T/Week in 4 Weeks — 0xJeff · 2026-11-03
- Claude Code caches are model-specific: switching models, compacting and idle time all bust the cache — shoumikgoswami · 2026-11-03
- Strata update doubles local inference speed for quad-R9700 setups: 70 to 140-160 tok/s — poofph · 2026-11-03
- Space-based distributed training gains traction as researchers push orbital datacenter vision — MillionInt · 2026-11-03
- Musk runs the numbers: space radiators dissipate ~1300 W/m², making orbital datacenters feasible — elonmusk · 2026-11-03