Ling MTP on DGX Spark: acceptance length rises with n, but prose throughput falls
niacolhealth · reddit · 2026-09-07
A corrected analysis of Ling 3.0-flash speculative decoding (MTP) benchmarks on one 128GB DGX Spark, filling in the missing no-speculation baseline.
Corrected numbers (official INT4 checkpoint, vendor vLLM fork, short coding task):
| Config | tok/s |
|---|---|
| Eager, MTP off | 20.8 |
| CUDA graphs, MTP off | 22.9 |
| CUDA graphs, MTP n=1 | 40.9 |
That's 10% from CUDA graphs, then 79% from MTP (different denominators). An earlier "two flags" result changed graphs and MTP together and couldn't attribute the gain—the key correction here.
MTP n=1/2/3: mean acceptance length 1.87 → 2.39 → 2.77, but prose 512-token output drops from 38.7 to 34.8 and 33.6 tok/s (2k output similar); code throughput stays flat. Acceptance length is not the optimization objective; n=1 was the useful setting for this checkpoint.
Caveats: the acceptance metric comes from a vendor fork with unknown counting; the timing denominator isn't fully specified, so treat numbers as reported throughput. The transferable method: isolate the no-MTP baseline first, then compare draft settings on your actual output types.
More from Infra
- Anthropic S-1 filing reveals 85% H1 gross margin and 5,300 employees — Jsevillamol · 2026-10-26
- HBM demand shifts: low-stack inference plans shelved, Rubin to go 12-high only — BenBajarin · 2026-10-26
- Anthropic generated $2.08 revenue per $1 of compute spend in Q2 2026, up from $0.63 — cedric_chee · 2026-10-26
- CircuitHub unveils The Grid, a factory-scale robot making PCBs in the US twice as fast as Shenzhen — garrytan · 2026-10-26
- Ubicloud publishes "Fixing Tool-Call Accuracy in vLLM" — petrusenko_max · 2026-10-26
- Meta approves nonvoting Class C shares as 2026 capex guidance hits $135B–$145B — thione · 2026-10-26