Interactive Speculative Decoding Tutorial for NeurIPS Explains When It Stays Lossless
Madisonkanna · x · 2026-09-07
Lily Zhang and Madison Kanna built an interactive tutorial for the NeurIPS 2026 Education Track on speculative decoding, the acceleration technique now running under nearly every hosted LLM.
- Why it matters: autoregressive decoding needs n forward passes through a tens-of-billions-parameter model to produce n tokens, making it the main inference bottleneck. Speculative decoding (Leviathan et al., 2023) has a lightweight draft model propose several tokens that the target model verifies.
- Losslessness: verified output preserves the target model's distribution exactly.
- Coverage: how the technique evolved, when it stays lossless, and what's next — including diffusion-inspired ideas for drafting many tokens at once instead of sequentially.
- Includes an interactive site and companion blog post.
More from Infra
- SurfSense: an open-source local NotebookLM with MCP agents and a prompt-caching lesson — FurtiveMirth · 2026-10-23
- Meta researcher: horrifying infra, not talent, is why OSS can't catch closed frontier — A_K_Nain · 2026-10-23
- Jane Street and NVIDIA speed up Mamba-2 with custom cuTile kernels — blelbach · 2026-10-23
- LatentLoop: asymmetric compute allocation cuts looped-MoE training cost by 19% — pmttyji · 2026-10-23
- Same AI request emits 8.5x more CO2 in Texas than in Sweden — jlundberg · 2026-10-23
- SK Hynix Pulls Plug on Solidigm US IPO as Chip Stocks Slide, Shares Down ~30% This Quarter — rohanpaul_ai · 2026-10-23