Uno paper: diffusion adapters on AR LLMs beat speculative decoding with lossless speedups
LucaAmb · x · 2026-11-10
A Kempner Institute blog post explains the Uno paper by Subham Sahoo's team: a way to combine diffusion-style parallel generation with autoregressive LLMs.
Core idea
- Keep the standard AR architecture; add a small set of diffusion weights per layer
- Diffusion weights draft a whole block of tokens in one forward pass; unchanged AR weights verify the draft
- Output follows the AR model's distribution exactly — lossless speedup
Results
- Faster than all speculative decoding methods (DFlash, EAGLE-3) at every tested batch size
- Beats all diffusion LLMs: Mercury 2, Diffusion Gemma, Llada
Context: token-by-token decoding is memory-bound, with GPUs mostly shuffling weights and KV cache; producing multiple correct tokens per pass improves both GPU utilization and throughput. Uno can be built on open-weight models like Qwen3-8B. A follow-up comparing DFlash2 and DSpark is coming.
More from Infra
- Rank #99 open-sources 5 H200 kernels from Open Kernel Challenge, up to 2.66x over SGLang — abhijithneil · 2026-11-10
- Paper quantifies on-policy distillation's tolerance to training-inference mismatch — Cong Yu · 2026-11-10
- Seroter's daily reading list: new TPUs, Claude 5.2 price cut, and the future of the PM role — rseroter · 2026-11-10
- DOE Funds $11.5M GridFM 2.0 AI Project Targeting 1B Grid Scenarios in 24 Hours — ingliguori · 2026-11-10
- NVIDIA open-sources Molt: a 9k-line RL framework where the whole setup is one Python file — algo_diver · 2026-11-10
- Microsoft Foundry adds Kling, LTX, and MiniMax video models via new fal partnership — gorkem · 2026-11-10