SOTA GGUF quants for Qwen3.8-Flash-Next: near-baseline quality at 1/5 the size
BullfrogScary8947 · reddit · 2026-09-16
ISTA-DASLab released GSQ-RCO GGUF quantizations of Qwen3.8-Flash-Next, cutting the 80-95GB model to 68-76GB with near-baseline quality. Key points:
- IQ3XXS is the strongest point: matches baseline exactly on AIME25 (100.00), within 0.51 on GPQA-Diamond and 1.14 on LiveCodeBench v6, at roughly one fifth of BF16 size.
- Q20 targets speed: avoiding lookup-table-heavy formats, it delivers 3.4x prompt throughput and 1.9x lower end-to-end latency vs IQ2XS, with 6.2x better prompt throughput in coding; decode rate stays flat across workloads. Trade-off: 89.07 vs 89.16 task average, 3.5 points below IQ3XXS.
Available on Hugging Face for local deployment, pick per throughput/quality needs.
Related event: ISTA-DASLab Releases Quantized Qwen3.8-Flash-Next GGUF(2 posts)→
More from Infra
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- Disable MTP for 20-30% faster local LLM inference on limited RAM — InternalMode8159 · 2026-11-02
- Kioxia up 216% in a year but down 68% from its June 2026 peak — zephyr_z9 · 2026-11-02
- Intel plunges 29%, worst day since the 1970s, wiping $12B off US government stake — emmanuelvivier · 2026-11-02
- Researcher hoards nearly 1TB of DRAM, refuses to sell a stick to data centers — kevinnbass · 2026-11-02
- Self-Hosting LLMs at a Law Firm: Infra and Data Governance Harder Than the Model — EducationalGroup7245 · 2026-11-02