DSpine explained: 5-layer draft model guesses 16 tokens in one forward pass
karminski3 · x · 2026-11-10
A technical breakdown of why DSpine lifts Qwen3.8-27B inference 3x:
- MTP bottleneck: the built-in 1-layer MTP drafts serially (3 steps per SGLang convention), 1.6x speedup, max 4 accepted tokens per round
- DSpine approach (descended from DFlash, diffusion-inspired): a 5-layer 2B draft model fills 15 positions in parallel per forward pass (1 anchor + 15 to guess); the 27B backbone verifies 16 tokens at once, with acceptance length up to 7.2 on GSM8K
- Adjacent causal injection: at every layer, a gated connection writes each position's current prediction into its right neighbor, propagating causal dependency through depth so parallel drafts stay coherent
- Shift cache: all 16×16=256 candidate-pair scores for adjacent positions are computed in one parallel pass; word selection is a table lookup
- Tencent's agent end-to-end benchmarks: SWE-bench Pro 3.19x, τ²-bench 2.87x, up to 3.55x on WorkBuddy vs DFlash2
Related event: Tencent's DSpine Speeds Up Qwen3.8-27B Inference 3x(3 posts)→
More from Infra
- Frontier lab compute shifts hard: pre-training falls from 67% to 7%, post-training/RL now 55% — akbirthko · 2026-11-10
- Study of 1,600 Data Centers Finds Each Adds ~1,000-1,300 Local Jobs Within 3-5 Years — RachelVT42 · 2026-11-10
- rakyll teases Agent Substrate as 'agentic compute fabric' ahead of KubeCon keynote — rakyll · 2026-11-10
- ngrok AI Gateway adds Liquid AI and 160+ LLM API models; tiny LFM3-1.7B pages for a thirsty plant — JosephJacks_ · 2026-11-10
- Together Compiler hits 2.7x throughput gains on Kimi K3 at same per-user speed — togethercompute · 2026-11-10
- ThumbFree: open-source offline voice typing with on-phone Parakeet, sub-second latency — Affectionate-Top7058 · 2026-11-10