Local Benchmarking of DeepSeek V4 Flash Quantizations: Q3 vs Q8
Spicy_mch4ggis · reddit · 2026-08-04
A developer benchmarked the Unsloth Q8 and Q3 xxs versions of DeepSeek V4 Flash on a 128GB VRAM setup.
The findings reveal that the Q3 version processes prompts 3.5x faster and decodes 2x faster compared to the offloaded Q8 version. The author is evaluating if the speed gains justify the quality loss, and also discusses technical details like speculative decoding, 200k context performance, and thinking budget configurations.
More from Infra
- GitHub rewrote the 800k-line Copilot runtime in Rust using AI agents, cutting CPU and memory by over 90% — bibryam · 2026-10-16
- NVIDIA quietly delists the entry-level RTX 5050 from its marketplace, following the RTX 5090 — chemist_slime · 2026-10-16
- Grok claims 816B MoE model runs locally on dual M5 Ultra Mac Studios via MLX — grok · 2026-10-16
- Same 1,000 Cloudflare Durable Objects: $4,050/month cut to $22 with one trick — threepointone · 2026-10-16
- NVIDIA ships FP4-quantized Qwen3.8-Flash for local use, trending on Hugging Face — nvidia · 2026-10-16
- Baidu's Virtual Head Attention tames the KV cache, FLOPs, quality triangle in long-context LLMs — baidu · 2026-10-16