Benchmarking DeepSeek V4 Flash Quantization
CoplanarDimension · reddit · 2026-07-18
This Reddit post tests the GGUF quantized versions of **DeepSeek V4 Flash** locally, comparing two llama.cpp implementations: the `fairydreaming/llama.cpp dsv4 fork` and the mainline `llama.cpp b10064`. ### Test Configuration - **Hardware**: Ryzen 5950X, 128GB DDR4, RTX 3090 24GB, Windows 11 Pro - **Models**: `DeepSeek-V4-Flash.IQ3_XXS-AS.gguf` and `DeepSeek-V4-Flash.IQ2_S.gguf` - **Unified Parameters**: `llama-server`, context 196608, flash-attn enabled, `--no-mmap`, KV cache using `q8_0` ### Main Results - On the **fork**, `IQ2_S` processed prompts slightly faster than `IQ3_XXS-AS` (about 7% faster on cold start), but in warm scenarios, their prompt throughput was nearly identical at around **254 tok/s**. - For **generation speed**, `IQ2_S` was consistently faster, beating `IQ3_XXS-AS` by about **18%~19%**. - The author concludes that `IQ3_XXS-AS` is actually the better sweet spot; while `IQ2_S` generates faster, the quality difference is more significant. ### Fork vs. Mainline Retesting the same `IQ3_XXS-AS` model on the mainline `b10064`: - **Cold start prompt**: Fork is faster (262.66 tok/s vs 227.38 tok/s) - **Generation speed**: Mainline is slightly faster (11.88 tok/s vs 11.42 tok/s) - In warm scenarios, the gap narrows, prompt speeds are basically equal, and generation remains slightly in favor of the mainline. The author's final takeaway is that this dsv4 fork is no longer necessary for these current tests.
Related event: Developers Benchmark DeepSeek Quantized Models Locally(2 posts)→
More from Infra
- More open models and llama.cpp updates are coming, says Merve Noyan — mervenoyann · 2026-07-21
- Why adding a second LLM provider breaks more than the API surface — Ok_Extension6373 · 2026-07-21
- UK AI datacentres face backlash over heat, noise and land use — nordicinst · 2026-07-21
- Fluidstack raises $830M at $7.5B valuation as Anthropic backs a $50B compute buildout — rohanpaul_ai · 2026-07-21
- Early Krea2 Gradio WebUI targets 6GB low-VRAM local runs — Fluid_Kaleidoscope17 · 2026-07-21
- Z.AI starts running a 1GW AI data center built entirely on domestic chips — Polymarket · 2026-07-21