Running Qwen 3.8 MoE on a single DGX Spark: A practical recipe
QuixiAI · x · 2026-08-29
Shantanu Goel shares a practical recipe for running the 135GB Qwen3.8-Flash-Next MoE model on a single NVIDIA DGX Spark (128GB VRAM). The solution uses NVFP4, MTP speculative decoding, and CUDA graphs. To fit the model, torch.fromfile is used to mmap the PLE table to NVMe, keeping it out of the unified memory pool while maintaining CUDA graph compatibility. The author also patched SGLang to bypass kernel gates restricting features to SM100, enabling FA4 paths on GB10 (SM121). Benchmarks were run in practical task configurations.
More from Infra
- Copper to optical: optical backplanes seen as necessary as racks move to optics — BenBajarin · 2026-10-16
- Polymarket prices 28% odds a US state enacts data center moratorium by year-end — Polymarket · 2026-10-16
- Pennsylvania residents reject $10,000-per-household offer for massive Amazon data center — Polymarket · 2026-10-16
- PyTorch now runs natively on AWS Trainium with zero code changes — PyTorch · 2026-10-16
- Chinese labs run on ~1/10 US compute; ByteDance pays 2-4x to hoover up SEA supply — robleclerc · 2026-10-16
- Zymtrace raises $12.2M and hires ML engineers to build agents that fix GPU bottlenecks — cfregly · 2026-10-16