Running DeepSeek-V4 1M Context on a Single RTX 5090 with vLLM
BlackBeardAI · reddit · 2026-08-04
A developer shared a detailed setup for running DeepSeek-V4-Flash with a full 1M context on a single RTX 5090 (32GB) paired with 256GB DDR5 RAM.
Deployment Details:
- Utilized vLLM 2.3.9 with CPU/RAM offloading, keeping most MoE experts in system RAM while retaining two complete routed MoE layers on the GPU.
- Achieved 800 tps prefill and 15+ tps decode speeds.
- Patched a底层 bug where FlashInfer's CUDA IPC hijacked TileLang.
Speculative Decoding:
- The author noted that DSpark speculative decoding behaves very differently during long reasoning sections. Draft acceptance can collapse to 30-50%, dropping generation speeds to 11-13 tok/s.
Related event: Single RTX 5090 Runs DeepSeek 1M Context(2 posts)→
More from Infra
- GitHub rewrote the 800k-line Copilot runtime in Rust using AI agents, cutting CPU and memory by over 90% — bibryam · 2026-10-16
- NVIDIA quietly delists the entry-level RTX 5050 from its marketplace, following the RTX 5090 — chemist_slime · 2026-10-16
- Grok claims 816B MoE model runs locally on dual M5 Ultra Mac Studios via MLX — grok · 2026-10-16
- Same 1,000 Cloudflare Durable Objects: $4,050/month cut to $22 with one trick — threepointone · 2026-10-16
- NVIDIA ships FP4-quantized Qwen3.8-Flash for local use, trending on Hugging Face — nvidia · 2026-10-16
- Baidu's Virtual Head Attention tames the KV cache, FLOPs, quality triangle in long-context LLMs — baidu · 2026-10-16