DeepSeek-V4-Flash Local Deployment Benchmarks: Hardware Tests and Quantization

Following the release of the DeepSeek-V4-Flash-0731 model, the developer community has seen a surge of local deployment and performance benchmarking results. Tests span from consumer-grade GPUs to personal AI supercomputers, proving that with proper quantization and deployment strategies, frontier LLMs can run highly efficiently on local machines. This significantly elevates the practical value of personal computing devices like the NVIDIA DGX Spark.

已确认

• DGX Spark 性能表现: A dual-machine DGX Spark setup using official FP8 weights achieved a single-stream throughput of 82 tok/s (peaking at 95 tok/s) and 135 tok/s at 3 concurrent requests. A single device ran the 155G model using 2-bit quantization and ranked first in 3bit quantization tests with 16.5 tok/s. With an 85% cache hit rate, the dual-machine setup can process up to 100 million input tokens daily.
• 专业与消费级显卡实测: On 2× RTX PRO 6000 Blackwell combined with speculative decoding, the median single-stream throughput reached 243 tok/s; this configuration can max out a million-token context (with concurrent capacity halved accordingly). The RTX PRO 6000 eGPU achieved 44-59.5 tok/s. Additionally, an A6000 + 256GB RAM setup ran at 17.2 t/s, and 3× RTX 3090 managed to run the Q3 quantized version using 119GB of VRAM.
• 量化与加速方案: For mixed multi-GPU setups, developers achieved a 40% decoding speedup using a custom GGUF approach that only reduces routing expert tensors to IQ3XXS. A single DGX Spark, leveraging vLLM-Moet 2-bit quantization and MTP acceleration, boosted aggregate throughput to 58.5 tok/s under 12 concurrent requests.

为什么重要

These extensive benchmark results shatter the stereotype that frontier LLMs rely solely on cloud computing. Through speculative decoding, mixed-precision quantization, and unified memory architectures, individual developers can not only prototype locally but also handle high-concurrency agent workloads, drastically lowering the token costs and barrier to entry for AI application development.

2026-08-02 ~ 2026-08-03 · 14 related posts

• DeepSeek Hits 243 tok/s on Dual RTX 6000 with Speculative Decoding — TheZachMueller · 2026-08-02
• [source] DeepSeek v4 Flash Hits 82 tok/s on Dual DGX Sparks — andrewchen · 2026-08-02
• Running DeepSeek V4-Flash Locally: Dual RTX 6000 Rig Handles Only Single User — dee_hw · 2026-08-02
• [source] DeepSeek V4 Flash 3-bit Quantization Tested on 3x RTX 3090: 119GB VRAM — consultkitapp · 2026-08-02
• DeepSeek's New Release Significantly Boosts the Value of Nvidia DGX Spark — firstadopter · 2026-08-02
• DeepSeek-V4-Flash hits 44-59.5 tok/s on RTX PRO 6000 eGPU with llama.cpp — backslashHH · 2026-08-02
• Custom IQ3 Requant for DeepSeek-V4-Flash: 1.4x Decode Speed on Mixed-GPU Rigs — HockeyDadNinja · 2026-08-02
• DeepSeek v4 Flash on Single DGX Spark Beats Dual-Spark Setup in Agentic Workflows — pbaylies · 2026-08-02
• [source] DeepSeek-V4 Local Test: 17 t/s on A6000 + 256GB RAM — USBhost · 2026-08-02
• Running Frontier Models Locally: DeepSeek V4 Flash on 2x DGX Sparks — lifebypixels · 2026-08-02
• Running DeepSeek V4-Flash with 1M Context on 2x RTX PRO 6000: Concurrency Up 1.5x — dee_hw · 2026-08-02
• Running Agentic Coding on Two DGX Sparks: 100M Tokens/Day Economics — HarveenChadha · 2026-08-02
• Running DeepSeek V4 Flash 155G on DGX Spark: 2-bit Quantization & MTP Benchmarks — Puzzleheaded_Base302 · 2026-08-03
• Running LLMs on DGX Spark: DeepSeek V4 Flash Tops the Ranking — andersonbcdefg · 2026-08-03