Running DeepSeek V4 Flash Locally: Achieving 700pp/s on a $1k Custom Rig
reto-wyss · reddit · 2026-08-03
A developer successfully deployed DeepSeek-V4-Flash-0731 locally on a workstation equipped with an RTX 5090 and 512GB of DDR4 RAM, sharing detailed performance benchmarks.
Using llama.cpp with the -b 8192 -ub 8192 --cpu-moe flags, the system maintained a prompt processing speed of 700 pp/s and a generation speed of 18 tg/s, even at a 100k token context length. The author noted that llama.cpp currently lacks native FP8 cache support for this model, doubling VRAM usage.
Furthermore, the author proposed a highly cost-effective DIY hardware build: pairing a used server motherboard and an EPYC 7002/7003 CPU with cheap DDR4 ECC memory, supplemented by mid-range consumer GPUs (like dual RTX 5060 Ti). This setup allows running massive MoE models locally for a base hardware cost of around $1,000.
More from Infra
- Minimax H3 Tested: Runs Locally on 8GB VRAM — inuptia · 2026-08-04
- Compute Scarcity vs. Creativity: Debating the Future of Neo AI Labs — reneeshah123 · 2026-08-04
- Self-Improving Agents Optimize vLLM, Boosting DeepSeek Throughput by 16% — yisongyue · 2026-08-04
- NVIDIA and KAIST Launch Joint AI Lab to Advance Agentic AI in Korea — hyunw_kim · 2026-08-04
- Running Frontier Models on 24GB VRAM: Local Deployment Challenges Cloud — mintybadgerme · 2026-08-04
- Self-Hosting AI Dev Environments: Sandboxing and Multi-Model Orchestration — Illhoon · 2026-08-04