Draft Model Proposes k Tokens, Verified in One Forward Pass
The series introduces the draft model, which proposes k candidate tokens cheaply, and shows that feeding the whole draft to the target model with causal masking yields all conditional probabilities in a single forward pass — though tokens cannot be accepted blindly.
2026-10-22 ~ 2026-10-22 · 2 related posts
- Speculative decoding: a small draft model proposes k tokens at low cost — cneuralnetwork · 2026-10-22
- One target pass scores all draft tokens — but blind acceptance breaks the distribution — cneuralnetwork · 2026-10-22