Single GH200 Runs DeepSeek V4 Flash: 2.7x Speedup with DSpark Offloading Guide
funding__secured · reddit · 2026-08-17
The author shared a complete guide for deploying DeepSeek-V4-Flash-0731 on a single NVIDIA GH200. Since the model (284B MoE) exceeds the 144GB HBM capacity, the author used vLLM's UVA (Unified Virtual Addressing) feature to offload 88GB of expert weights to the 480GB LPDDR5x system memory, combined with DSpark speculative decoding.
Key Configuration & Results:
- Utilized vLLM nightly build with native DSpark support, requiring zero patches.
- Enabled cpu-offload-gb 88 and offload-backend uva to handle memory constraints.
- Performance: Baseline 64 tok/s → 171 tok/s with DSpark (k=5), achieving a 2.7x speedup.
Gotchas:
- Nightly build is required; v0.26.0 is incompatible.
- Only offload expert weights; offloading FP8 attention weights breaks DeepGEMM.
- Selective expert streaming is unnecessary due to fast NVLink C2C bandwidth.
More from Infra
- Phison CEO calls CXMT memory scare a 'ghost story', warns of materials shortage — BenBajarin · 2026-10-16
- Ben Bajarin: The Future of Compute Is Fungible as Hyperscalers Design for Workload Flexibility — BenBajarin · 2026-10-16
- AirLLM runs frontier-size models layer-by-layer: 671B on 12GB VRAM — Aiden_Tech_Ai · 2026-10-16
- NVIDIA Launches cuPotential in CUDA-X to Bring AI Agents to Semiconductor Materials Simulation — tomaszbednarz · 2026-10-16
- Q4-quantized coder model on 8-core CPU builds interactive solar system demo in one shot — IntrepidMindExplorer · 2026-10-16
- TokenSpeed MLA 2 ships for Vera Rubin: up to 3.44× faster Kimi K3 decode — zhyncs42 · 2026-10-16