Tencent's DSpine speculative decoding ships GGUFs and a llama.cpp PR: 4.5x speedup on 27B
pmttyji · reddit · 2026-11-17
Tencent's DSpine speculative decoding (parallel drafting, depth conditioning) is now practical: Qwen3.8-27B DSpine b16 GGUFs are on Hugging Face with model cards comparing against DFlash2, and llama.cpp PR #31941 is open. On an RTX 3090 24GB: 125 tok/s on code/math (4.5x vs no speculation, 2x vs DFlash2), 90 tok/s on mixed chat (3.2x); max context 64k at 15 drafted, 98k at 7. Advantage holds at depth: 35.5 tok/s at 64k vs 19.5 without.
More from Infra
- Pipeshift claims 40% faster first token than Sarvam serving Gemma 4 31B at 150B tokens/day — NirantK · 2026-11-17
- LoopFill: training-free speculative decoding speeds up looped Transformers — UIUC-CS · 2026-11-17
- Founder lands 1000 GPUs (40% GB300, 60% B300) on short-term contracts, dismissing the GPU shortage narrative — SumitGup · 2026-11-17
- Agent Substrate: open-source zero-trust runtime suspends idle agents, fits 10x more per hardware — Saboo_Shubham_ · 2026-11-17
- Practical tips to cut agent memory costs on OpenAI: cache TTL, Flex, fewer calls — gabrielchua · 2026-11-17
- xAI affiliate buys 96 acres near Memphis Colossus 2 for $5.5M as compute footprint expands — XFreeze · 2026-11-17