Qwen3.8-Flash-Next on 2x DGX Spark NVFP4: 50 t/s decode, 2,900 t/s prefill

-dysangel- · reddit · 2026-08-30

Redditor -dysangel- shares a full config for running Qwen3.8-Flash-Next NVFP4 on 2x NVIDIA DGX Spark (GB10): 49.7 t/s decode on structured output / 34.8 t/s prose, 2,875 t/s prefill on 11k tokens under TP2; single node does 35 t/s.

Stack highlights:

vLLM PR #53896 release/qwen38next branch (model absent from main) plus a 2-file sm121 patch: add arch 12.1 to CMake lists and a 90-line software E2M1 conversion (GB10 lacks the hardware cvt instruction)
48GB PLE n-gram table served from internal NVMe via mmap — MADVRANDOM makes a 30x difference on hash-scattered lookups; a node-local CPU worker does gathers over ZMQ + pinned buffers + CUDA-IPC
eager + MTP k=3 in a native venv, not Docker

Dead ends documented:

PIEWISE cudagraphs cost 28% decode under MTP+TP2 (36 vs 50 t/s); they help single-node no-MTP (+80%)
Docker silently degrades NCCL to TCP sockets (40% prefill); needs --privileged or IB passthrough
RoCE device names are unstable across reboots and differ between identical machines; enumerate with ibdev2netdev and verify NET/IB via NCCLDEBUG=INFO
flashinfer must be 0.6.18 on GB10; 0.6.17 crashes the NVFP4 MoE fallback kernel
vLLM's shuffleInputRowsKernel has an uninitialized-permutation OOB read masquerading as a CUTLASS GEMM failure; a two-line guard fixes it

Next steps: block-diffusion drafting (DFlash/DSpark) and disaggregated decode onto a big-bandwidth Mac. MTP acceptance is 3.8 tokens/step on structured output but noticeably lower on prose — benchmark both.

Original post →

Qwen3.8-Flash-Next on 2x DGX Spark NVFP4: 50 t/s decode, 2,900 t/s prefill

More from Infra