Tencent's DSpine lifts speculative decoding acceptance length 27.8% over DFlash
pmttyji · reddit · 2026-11-07
Tencent released DSpine, an open-source drafter for parallel speculative decoding.
- Problem: parallel drafting selects tokens independently, producing inconsistent continuations that shorten the accepted prefix; prior methods handle conditioning in a lightweight module after the backbone.
- Approach: gated adjacent injection at every layer writes each predecessor's predicted feature into its successor, unfolding the causal conditioning chain across depth while all positions update in parallel, with a unified transfer space built from the target model's output embeddings and layer-wise output-embedding supervision.
- Results: longest acceptance length across seven math/code/chat benchmarks on Qwen3-4B and Qwen3-8B. At temperature 0 on Qwen3-8B, mean acceptance length rises from DFlash's 3.77 to 4.82 (+27.8%); SGLang serving throughput is 23.3% higher than DFlash on average.
- Implemented with fused kernels and a transition cache in SGLang; paper, code, and weights are all available.
Related event: Tencent Open-Sources DSpine Draft Model, Speeding Up 27B Inference by 3x(4 posts)→
More from Infra
- Data-center water use debate: a post misunderstanding basic water science gets 22K likes — ATTlKA · 2026-11-10
- Qdrant presents vector index-native relevance feedback in production at CIKM Rome — qdrant_engine · 2026-11-10
- int8 quantization plus fp16 rescoring: 1.4M embeddings retrieved in under 2ms — tomaarsen · 2026-11-10
- int8 embeddings plus Usearch index: 1.4M vectors searched in under 2ms on CPU — tomaarsen · 2026-11-10
- Google rolls out broader AI chip lineup to challenge Nvidia in a market worth hundreds of billions — dinabass · 2026-11-10
- Thinking Machines engineer returns to Modal, reflecting on a year of frontier ML infrastructure work — josh_wills · 2026-11-10