Running Qwen3.8 Flash NVFP4 on a single DGX Spark: 1M context at 37 tok/s
BLUECOW009 · x · 2026-09-04
Mia AI Lab published a self-contained recipe for serving the 99 GB Qwen3.8-Flash-Next NVFP4 checkpoint on a single DGX Spark (121 GiB unified memory, TP=1) via vLLM with the PLE table offloaded and memory-mapped.
Measured performance:
- Up to 1M context with a 1,431,164-entry KV cache
- Full text, image, and video support out of the box
- 37 decode tok/s single stream, up to 86 tok/s with 4 concurrent streams
- 1500-2000 tok/s prefill; passed a 400k prefill stress test
The author considers it the best model to run on one DGX Spark, beating Qwen3.8-27B and the single-Spark DeepSeek v4 Flash. The repo ships start/stop scripts (10-12 min boot) and documents pitfalls like /dev/shm segment leaks.
More from Infra
- Spotify's Portal Cut Claude Code Token Usage by 90% With a Two-Mode Router — rseroter · 2026-09-04
- Vyact: Open-Source Desktop Workspace Unifying Local LLMs, RAG, and Browser Context — vyact · 2026-09-04
- At what context depth does KV quantization start to hurt? An F16 vs Q8/Q4 parity PoC — Slight_Analysis_5414 · 2026-09-04
- KV caching: the fundamental optimization behind autoregressive LLM inference — alec_helbling · 2026-09-04
- WSJ: Data centers are a "lottery win" for workers in Richland Parish, Louisiana — robleclerc · 2026-09-04
- No orchestrator, no MCP: 4 peer agents on a gossip mesh research the web and deliver to Slack — Trainer_Intelligent · 2026-09-04