Block Diffusion (ICLR 2025 Oral) Marries Parallel Generation With KV Caching
alec_helbling · x · 2026-09-11
Block Diffusion (arXiv:2503.09573, ICLR 2025 Oral) interpolates between autoregressive and diffusion LMs. Diffusion LMs generate tokens in parallel but iterative unmasking updates token states, killing KV-cache reuse. Block Diffusion decodes blocks left-to-right while generating each block in parallel, restoring efficient caching and enabling flexible-length generation. The paper adds a recipe: efficient training, gradient-variance estimators, and data-driven noise schedules. It sets a new SOTA among diffusion models on LM benchmarks, with code and weights open-sourced.
Related event: Block Diffusion Brings KV-Cache Reuse to Diffusion LMs(2 posts)→
More from Research
- Rachel Thomas points to classic paper on articulating the value of not building tech — math_rachel · 2026-10-28
- Meta's Omnilingual MT covers 1,600 languages, 1B–8B models rival 70B LLMs — kalyan_kpl · 2026-10-28
- Supercharging LLM classification with logprobs: soft classifiers you can threshold-tune — JnBrymn · 2026-10-28
- cryoET Data Portal releases three datasets benchmarking crossed laser phase plate — ZhongingAlong · 2026-10-28
- Blind Robot Hand Spins Baoding Balls 225 Times With Only Touch — Scobleizer · 2026-10-28
- Rollback: World Models Rewind Robot Decisions to Explore Counterfactual Actions — agibot-world · 2026-10-28