Ling MTP on DGX Spark: acceptance length rises with n, but prose throughput falls
niacolhealth · reddit · 2026-09-07
A corrected analysis of Ling 3.0-flash speculative decoding (MTP) benchmarks on one 128GB DGX Spark, filling in the missing no-speculation baseline.
Corrected numbers (official INT4 checkpoint, vendor vLLM fork, short coding task):
| Config | tok/s |
|---|---|
| Eager, MTP off | 20.8 |
| CUDA graphs, MTP off | 22.9 |
| CUDA graphs, MTP n=1 | 40.9 |
That's 10% from CUDA graphs, then 79% from MTP (different denominators). An earlier "two flags" result changed graphs and MTP together and couldn't attribute the gain—the key correction here.
MTP n=1/2/3: mean acceptance length 1.87 → 2.39 → 2.77, but prose 512-token output drops from 38.7 to 34.8 and 33.6 tok/s (2k output similar); code throughput stays flat. Acceptance length is not the optimization objective; n=1 was the useful setting for this checkpoint.
Caveats: the acceptance metric comes from a vendor fork with unknown counting; the timing denominator isn't fully specified, so treat numbers as reported throughput. The transferable method: isolate the no-MTP baseline first, then compare draft settings on your actual output types.
More from Infra
- Musk: robot RAM demand will dwarf everything else, and 100% of superintelligence compute goes to deep space — elonmusk · 2026-10-27
- RTX 5090 hits $7K as PC gaming community goes into 'denial' over AI boom — PAstynome · 2026-10-27
- Wan2GP lands on Pinokio: 1-click AI video on 6GB GPUs — cocktailpeanut · 2026-10-27
- AWS on agent prompt caching: explicit + implicit patterns to slash token costs — AWS ML Blog · 2026-10-27
- Thieves spoofing GPS steal $100M+ in data center cargo since June — Ars Technica AI · 2026-10-27
- Lam Research: weekly token usage up ~250x in 18 months — Beth_Kindig · 2026-10-27