SGLang DSpark reaches 1.89× decode throughput on 8×B200
BanghuaZ · x · 2026-07-23
A DSpark speculator for Inkling NVFP4 was trained end-to-end with SpecForge on a live SGLang target engine.
- 1.89× decode throughput vs. a non-speculative baseline at batch size 64 on 8×B200 with TP8
- 7–14% faster than Inkling’s built-in MTP at the same batch size
- 3.66 average accept length, reaching 4.76 on GSM8K
Implementation details:
- A 5-layer Qwen3-style DFlash parallel-draft backbone
- A rank-256 Markov logit-bias head for intra-block dependencies
- A per-position confidence head to predict acceptance
- Distillation from 400K Inkling regenerations
The post invites readers to try it, with the serving command shared in the comments.
Related event: Open-Sourced DSpark Speculator Boosts Inkling Throughput by 1.89x(3 posts)→
More from Infra
- MiniMax H3 runs fully on a Galaxy S26: 5-second video with audio in ~7 minutes — BidFunny4862 · 2026-10-27
- Unconventional AI acquires inference chip firm NeuReality in 1,000x efficiency bid — NaveenGRao · 2026-10-27
- Musk: robot RAM demand will dwarf everything else, and 100% of superintelligence compute goes to deep space — elonmusk · 2026-10-27
- RTX 5090 hits $7K as PC gaming community goes into 'denial' over AI boom — PAstynome · 2026-10-27
- Wan2GP lands on Pinokio: 1-click AI video on 6GB GPUs — cocktailpeanut · 2026-10-27
- AWS on agent prompt caching: explicit + implicit patterns to slash token costs — AWS ML Blog · 2026-10-27