Running DeepSeek at 1M Context on Single RTX 5090 via Adaptive Dual CUDA Graphs
BlackBeardAI · reddit · 2026-08-05
A developer shared the latest optimizations for running the massive DeepSeek-V4-Flash-0731 (155GB) on a single RTX 5090 while maintaining its native 1M token context.
Previously, fixed speculative decoding depth caused unstable throughput due to varying acceptance rates between hard reasoning and code generation. To fix this, the author developed a phase-adaptive mechanism:
- Switches to K=1 during the <think> reasoning phase.
- Switches back to K=2 for final content/code generation.
To prevent performance drops from mismatched CUDA graph shapes during switching, the author implemented dual CUDA graph capture. Ultimately, at the full 1M context, reasoning throughput reached 13.8 tok/s and code generation hit 17.0 tok/s.
Related event: Single RTX 5090 Runs DeepSeek 1M Context(2 posts)→
More from Infra
- Mixedbread on retrieval scaling laws: co-designing models and vector DBs — lateinteraction · 2026-08-26
- Data Center Backlash Not Driven by Anti-Tech Sentiment — AndyMasley · 2026-08-26
- AI Agent Security Market: Can Zscaler Become the Default Control Plane? — thedealdirector · 2026-08-26
- Running Qwen 27B on RTX 3060+2060 Yields Only 5-6 TPS — sheriffoftiltover · 2026-08-26
- PyTorch PR fixes static specialization for FSDP modules — ezyang · 2026-08-26
- Inference Spend Isn't Speculative: Why AI Tokens Differ From Dot-Com Hardware Hoarding — AccBalanced · 2026-08-26