Local quant bake-off: 129GB Qwen3.8 Flash scores 25/25, beats 390GB DeepSeek V4.1
whiteh4cker · reddit · 2026-10-23
A Reddit user benchmarked local quants: Qwen3.8-Flash-Next W4A16-FP8PLE (129GB) vs their own 2.6-bit DeepSeek-V4.1-Flash-1017 (187GB + 203GB Engram). Using a modified BenchKit, DeepSeek scored 23/25 while Qwen hit a perfect 25/25. Notable cases: DeepSeek doom-looped for 128s on HumanEval #22 and timed out at 298s on #24, while Qwen answered correctly in 18.8s and 34.5s. Small sample, but the takeaway for local deployment: a smaller high-quality quant can beat an ultra-low-bit large model.
More from Infra
- SK Hyper in talks to fund envisioned $350B AI data center buildout in South Korea — dinabass · 2026-10-23
- MiniMax M3.1 Flash hits >1K tok/s decode with day-0 SGLang support across NVIDIA and AMD — MiniMax_AI · 2026-10-23
- MiniMax-M3.1-Flash goes open weights with day-0 NVIDIA NeMo fine-tuning up to 128 GPUs — MiniMax_AI · 2026-10-23
- RadixArk caught paying KOLs to coordinate quote-posts of SGLang announcement — EugeneVinitsky · 2026-10-23
- MiniMax partners with vLLM creators' Inferact for day-zero production inference of open models — woosuk_k · 2026-10-23
- Local LLM shootout: Flint 40B MoE vs Qwen3.8-27B on identical RTX A6000s — RichardSocher · 2026-10-23