DeepSeek V4.1 Flash runs at 63.5 tok/s on a single RTX 5090 rig
jesdga95 · reddit · 2026-10-21
A developer got DeepSeek V4.1 Flash 1017 (Unsloth UD-IQ1S, 243GB) running in his own Basalt inference engine on an RTX 5090 + RTX 5060 Ti 16GB + 128GB DDR5 setup.
Method
- All 107 GiB of experts in RAM, 5090 as hot-expert cache; later offloading some experts to the 5060 Ti for extra speed. Engram table on SSD.
- Expert lookups: 66% 5090, 16% 5060 Ti, 5% PCIe copies, 12% CPU.
Results (vs 20 tok/s with llama.cpp --fit)
- 53.6 tok/s decode with experts in RAM; 63.5 tok/s with the 5060 Ti computing its own experts; prefill 610 tok/s.
- Tool calls, images, and up to 1M context work (a 1M prompt takes 26 min).
- Greedy decoding matches llama.cpp on all test prompts; even at IQ1S it passes most regular benchmarks with recall well above Qwen 3.8 FN.
Limits: the 12% CPU-bound lookups are bottlenecked by dual-channel DDR5; speculative decoding only 1.1–1.2x. Author suggests 96GB+ VRAM or quad/octa-channel DDR5 rigs as the sweet spot.
More from Infra
- A beginner-written CUDA guide that finally builds the full mental picture — goyal__pramod · 2026-10-22
- Shadeform adds CPU compute today, Apple Silicon instances coming this quarter — socialwithaayan · 2026-10-22
- RTX 3090 ships half-speed fp32 tensor math; gpu-unlock module boosts BF16 throughput 1.9x — sn2006gy · 2026-10-22
- Redditor converts on Qwen 3.8 Flash Next: quantized local run rivals last year's pro models — sshwifty · 2026-10-21
- Vercel's durable-harness moves coding agents to the cloud with auto-resume for month-long runs — mitsuhiko · 2026-10-21
- 256GB Mac Studio Beats DGX Spark by ~2x Running Qwen3.8-Flash Agent — GabGarrett · 2026-10-21