Running Qwen3.8 Flash NVFP4 on a single DGX Spark: 1M context at 37 tok/s
BLUECOW009 · x · 2026-09-04
Mia AI Lab published a self-contained recipe for serving the 99 GB Qwen3.8-Flash-Next NVFP4 checkpoint on a single DGX Spark (121 GiB unified memory, TP=1) via vLLM with the PLE table offloaded and memory-mapped.
Measured performance:
- Up to 1M context with a 1,431,164-entry KV cache
- Full text, image, and video support out of the box
- 37 decode tok/s single stream, up to 86 tok/s with 4 concurrent streams
- 1500-2000 tok/s prefill; passed a 400k prefill stress test
The author considers it the best model to run on one DGX Spark, beating Qwen3.8-27B and the single-Spark DeepSeek v4 Flash. The repo ships start/stop scripts (10-12 min boot) and documents pitfalls like /dev/shm segment leaks.
More from Infra
- PostgreSQL 19 slips to November as LLM-assisted code review floods committers with edge cases — sanjaykalra · 2026-10-21
- Decision Models' routing classifier hits AntSeed, cutting inference cost to 24% of always-Opus — airesearch12 · 2026-10-21
- Eon cuts engineer onboarding from 2 days to 2 minutes with an MCP Gateway on Bedrock AgentCore — AWS ML Blog · 2026-10-20
- a16z's Ben Horowitz says AI buildout isn't the 90s bubble — an investor pushes back — henloitsjoyce · 2026-10-20
- Micron and Nvidia drive ~36% of S&P 500's Q3 earnings growth — Beth_Kindig · 2026-10-20
- NVIDIA lands in Europe: Jensen Huang and German chancellor discuss industrial AI at GTC Berlin — nvidia · 2026-10-20