ReaDy Speculative Decoding Hits 3.16x Speedup on Qwen3.8-27B With a Single B200
pmttyji · reddit · 2026-10-23
The ReaDy paper is the first speculative decoding method to tap hybrid linear attention (HLA) recurrent states for drafting: each draft position uses a learned query to retrieve context from the target model's committed recurrent states, injected via a lightweight state reader with low-rank reads and fused operators to keep overhead low. Across seven benchmarks (math, code, dialogue), Qwen3.8-27B on a single NVIDIA B200 averages a 3.16x speedup over autoregressive decoding (up to 4.08x on GSM8K), beating DSpark on every benchmark with +7.97% throughput and +8.50% acceptance length on average. Code and weights are open-sourced on GitHub and HuggingFace.
More from Infra
- Meta researcher: horrifying infra, not talent, is why OSS can't catch closed frontier — A_K_Nain · 2026-10-23
- Jane Street and NVIDIA speed up Mamba-2 with custom cuTile kernels — blelbach · 2026-10-23
- LatentLoop: asymmetric compute allocation cuts looped-MoE training cost by 19% — pmttyji · 2026-10-23
- Same AI request emits 8.5x more CO2 in Texas than in Sweden — jlundberg · 2026-10-23
- SK Hynix Pulls Plug on Solidigm US IPO as Chip Stocks Slide, Shares Down ~30% This Quarter — rohanpaul_ai · 2026-10-23
- Designer mocks Nvidia: buy idle GPUs or lose access to new ones—definitely not a bubble — michalmalewicz · 2026-10-23