Ling MTP on DGX Spark: acceptance length rises with n, but prose throughput falls
niacolhealth · reddit · 2026-09-07
A corrected analysis of Ling 3.0-flash speculative decoding (MTP) benchmarks on one 128GB DGX Spark, filling in the missing no-speculation baseline.
Corrected numbers (official INT4 checkpoint, vendor vLLM fork, short coding task):
| Config | tok/s |
|---|---|
| Eager, MTP off | 20.8 |
| CUDA graphs, MTP off | 22.9 |
| CUDA graphs, MTP n=1 | 40.9 |
That's 10% from CUDA graphs, then 79% from MTP (different denominators). An earlier "two flags" result changed graphs and MTP together and couldn't attribute the gain—the key correction here.
MTP n=1/2/3: mean acceptance length 1.87 → 2.39 → 2.77, but prose 512-token output drops from 38.7 to 34.8 and 33.6 tok/s (2k output similar); code throughput stays flat. Acceptance length is not the optimization objective; n=1 was the useful setting for this checkpoint.
Caveats: the acceptance metric comes from a vendor fork with unknown counting; the timing denominator isn't fully specified, so treat numbers as reported throughput. The transferable method: isolate the no-MTP baseline first, then compare draft settings on your actual output types.
More from Infra
- NVIDIA's new Sol-H3 fast inference method for H3 awaits a ComfyUI port — krigeta1 · 2026-09-08
- Walking the AI rack optical stack: InP substrates and silicon photonics as the cleaner bet — demian_ai · 2026-09-08
- Running dual RX 7900 XTX on X570/X870 Taichi for local LLM inference: is x8/x8 enough? — espece-de-bon · 2026-09-08
- Hugging Face teases WebGPU inference engine with 5-10x speedups on Transformers.js — nicodotdev · 2026-09-08
- Meta to deep-dive recommendation inference systems at PyTorch Conference 2026 — PyTorch · 2026-09-08
- Memory crunch hits home: 4TB portable SSD prices stun as AI reprices the storage stack — demian_ai · 2026-09-08