Running Qwen 3.8 MoE on a single DGX Spark: A practical recipe
QuixiAI · x · 2026-08-29
Shantanu Goel shares a practical recipe for running the 135GB Qwen3.8-Flash-Next MoE model on a single NVIDIA DGX Spark (128GB VRAM). The solution uses NVFP4, MTP speculative decoding, and CUDA graphs. To fit the model, torch.fromfile is used to mmap the PLE table to NVMe, keeping it out of the unified memory pool while maintaining CUDA graph compatibility. The author also patched SGLang to bypass kernel gates restricting features to SM100, enabling FA4 paths on GB10 (SM121). Benchmarks were run in practical task configurations.
More from Infra
- engy.ai processes 50B+ tokens in a day as open-model inference demand surges — const_reborn · 2026-10-16
- New inference engine 'e' hits 10,500 tok/s on a single DGX Station — brandon_galang · 2026-10-16
- SGLang ships Day-0 support for FLUX 3 Image Klein: 10 reference images, BF16/FP8/NVFP4 variants — ying11231 · 2026-10-16
- NVIDIA's Nemotron 3.5 Super VL hits CoreWeave: 12B-active multimodal MoE at $0.25/M input tokens — wandb · 2026-10-16
- Chip equipment beats bull case: 2026 WFE seen at $153B, $278B by 2030 — BenBajarin · 2026-10-16
- Steam now launches in 5.23s, beating Epic's 5.47s — and vLLM gets roasted for slow startup — AlpinDale · 2026-10-16