DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
Xin Cheng, Xingkai Yu, Chenze Shao, Jiashi Li, Yunfan Xiong, Yi Qian, Jiaqi Zhu, Shirong Ma, Xiaokang Zhang, Jiasheng Ye, Qinyu Chen, Chengqi Deng, Jiping Yu, Damai Dai, Zhengyan Zhang, Yixuan Wei, Yixuan Tan, Wenkai Yang, Runxin Xu, Yu Wu, Zhean Xu, Xuanyu Wang, Muyang Chen, Rui Tian, Xiao Bi, Zhewen Hao, Shaoyuan Chen, Huanqi Cao, Wentao Zhang, Anyi Xu, Huishuai Zhang, Dongyan Zhao, Wenfeng Liang
cs.AI, cs.CL
2026-07-06
DSpark unifies semi-autoregressive drafting with load-aware verification; its accepted length beats Eagle3 by about 30% offline, and in live DeepSeek-V4 traffic it speeds per-user generation 60-85% at matched throughput.
Speculative decoding speeds up LLM inference by letting a small draft model guess several tokens and a big model verify them in one pass. Parallel drafters, which emit a long sequence in a single forward pass, have two flaws: tokens lack mutual dependency, so acceptance decays fast at later positions, and verifying whole blocks indiscriminately burns batch capacity on tokens likely to be rejected, which collapses throughput under high concurrency. DSpark fixes both, and it is already live in DeepSeek-V4 serving.
The draft side is a semi-autoregressive architecture. A heavy parallel backbone (DFlash) produces all hidden states in one forward pass, then a lightweight sequential module (Markov head or RNN head) adds transition biases conditioned on previously sampled tokens. The Markov head uses a low-rank factorization (r=256) to add first-order dependency, curbing suffix decay while keeping the sequential cost far below the parallel backbone.
The verification side is confidence-scheduled verification. A confidence head gives each token a conditional acceptance probability ck, supervised by the analytical ck = 1 − ½‖pdraft − ptarget‖₁ (the L1 distance between draft and target distributions). On top sits a hardware-aware prefix scheduler: it globally sorts candidate tokens by survival probability a = ∏ ci, then picks verification length per request from the engine's real-time throughput curve, to maximize throughput Θ.
Offline, DSpark's accepted length τ per round leads across the board:
| Target / task | DSpark | DFlash | Eagle3 |
| Qwen3-4B math | 5.57 | 4.80 | 4.56 |
| Qwen3-8B code | 5.42 | 4.80 | 4.15 |
| Qwen3-14B chat | 3.47 | 2.99 | 2.52 |
That is about 30% over the autoregressive Eagle3 and 17-18% over the parallel DFlash. The production result is more telling: in the DeepSeek-V4 system under live traffic, V4-Flash speeds per-user generation 60-85% at matched throughput and V4-Pro 57-78%. Under strict interactivity SLAs (V4-Flash at 80 tok/s/user, V4-Pro at 35 tok/s/user), aggregate throughput rises 51% and 52% over the MTP-1 baseline, pushing the serving system's Pareto frontier into territory it could not reach before.
Inference cost is the crux of LLM commercialization, and DeepSeek is an open engineering benchmark here. DSpark's value is optimizing draft quality and verification scheduling together and proving it on real high-concurrency traffic, not just offline acceptance. For inference-serving teams, semi-autoregressive drafting plus load-aware verification is a concrete, borrowable pattern.
The authors flag a fixed cost: generating the initial γ-token block on the draft side cannot be skipped, and for hard queries with naturally low acceptance that upfront compute is unrecoverable. They leave difficulty-aware early exiting to future work. The live gains are also measured on DeepSeek's own system (MTP baseline, specific hardware); moving to another engine means re-calibrating the throughput curves behind confidence scheduling, and that migration cost is not discussed.