DGX Spark MoE Inference: MTP Speculative Decoding Delivers 185% Throughput Gain
pmttyji · reddit · 2026-10-28
A practical guide stacks three optimizations for MoE inference on NVIDIA DGX Spark: the Humming MXFP4 MoE backend (+9.3%, 39.45 tok/s), online FP8 LM head quantization (+21.7% cumulative, 43.94 tok/s), and MTP speculative decoding with NEXTN (steps=3, topk=1, drafttokens=4), which cuts TPOT to 9.72ms and reaches 102.88 tok/s—185% above the 36.09 tok/s baseline.
More from Infra
- Netflix runs observability on RDF/OWL/SHACL ontology, 38M log events per second — JeremyCMorgan · 2026-10-28
- Reproducible perf benchmarks made easy with the hf CLI skill and HF Jobs — victormustar · 2026-10-28
- Local NCM benchmarks: 4x faster for one request, 4-8x slower at 64 concurrent — Prompt Engineering · 2026-10-28
- TinyLeague drops Artificial Analysis scores, adds KV cache efficiency ratings for local models — nunodonato · 2026-10-28
- Can DeepSeek v4.1 actually run at 1-3 bpw? Reddit locals debate ultra-low quant usability — Plastic-Stress-6468 · 2026-10-28
- Strata on 16GB VRAM: slow prefill at 162 tok/s as auto-tuning arms get skipped for memory — ckplscz · 2026-10-28