Running DeepSeek at 1M Context on Single RTX 5090 via Adaptive Dual CUDA Graphs
BlackBeardAI · reddit · 2026-08-05
A developer shared the latest optimizations for running the massive DeepSeek-V4-Flash-0731 (155GB) on a single RTX 5090 while maintaining its native 1M token context.
Previously, fixed speculative decoding depth caused unstable throughput due to varying acceptance rates between hard reasoning and code generation. To fix this, the author developed a phase-adaptive mechanism:
- Switches to K=1 during the <think> reasoning phase.
- Switches back to K=2 for final content/code generation.
To prevent performance drops from mismatched CUDA graph shapes during switching, the author implemented dual CUDA graph capture. Ultimately, at the full 1M context, reasoning throughput reached 13.8 tok/s and code generation hit 17.0 tok/s.
Related event: Single RTX 5090 Runs DeepSeek 1M Context(2 posts)→
More from Infra
- GitHub rewrote the 800k-line Copilot runtime in Rust using AI agents, cutting CPU and memory by over 90% — bibryam · 2026-10-16
- NVIDIA quietly delists the entry-level RTX 5050 from its marketplace, following the RTX 5090 — chemist_slime · 2026-10-16
- Grok claims 816B MoE model runs locally on dual M5 Ultra Mac Studios via MLX — grok · 2026-10-16
- Same 1,000 Cloudflare Durable Objects: $4,050/month cut to $22 with one trick — threepointone · 2026-10-16
- NVIDIA ships FP4-quantized Qwen3.8-Flash for local use, trending on Hugging Face — nvidia · 2026-10-16
- Baidu's Virtual Head Attention tames the KV cache, FLOPs, quality triangle in long-context LLMs — baidu · 2026-10-16