Local quant benchmark: new releases trail months-old DeepSeek/Qwen models
EmilPi · reddit · 2026-10-24
Testing GGUF quants on 2×RTX 3090 + 128GB RAM, the author ranks new models by Delta score: DeepSeek-V4-Flash-Vision-Exp (MXFP4) leads at 2.86, followed by Qwen3.8-Flash-Next variants. Tasks include agentic coding, reasoning, spatial, physics, adversarial and needle-in-haystack. Key findings: none of the new models beat months-old DS/Qwen releases; Qwen3.8-Flash-Next with Strata (-kv int8) loses no quality and runs faster; Nemotron-3.5-Bolt's QAT quant disappoints; Ling-3.0-Flash-Sante got faster but not better; K2-Horizon-7B is solid for its size.
More from Infra
- What $70-100K of hardware buys for local GLM5.3 and DeepSeek with 10-20 concurrent users — whoami-233 · 2026-10-24
- Stripe reportedly buying OpenRouter for ~$7.5B as AI usage billing becomes the prize — FinanceYF5 · 2026-10-24
- Redditor pushes back on local AI FUD: real inference speeds are 5x what the video shows — wojtek15 · 2026-10-24
- NVIDIA ships BioNeMo Inference Runtime v0.2 with RFD4-Proteina support — AllThingsApx · 2026-10-24
- ISTA-DASLab ships GSQ-RCO quantized DeepSeek-V4.1-Flash GGUFs, llama.cpp support still pending — Gold_Coconut9777 · 2026-10-24
- Sauers_ Mocks OpenAI Over Unstudied Multi-Agent Hardware — Sauers_ · 2026-10-24