Do ISTA's 2-bit GSQ-RCO DeepSeek quants really match the base model? Community asks for KLD
Queasy_Back2588 · reddit · 2026-10-25
A Reddit user questions ISTA-DASLab's GSQ-RCO quantized GGUFs of DeepSeek-V4.1-Flash, whose charts show IQ2XXS (2.0 bpw) matching the base model and even IQ1M (1.75 bpw) retaining 98.7% of the base task average, with AIME25 at 100 for every version.
The concerns:
- The benchmarks are saturated (90%+ across the board, AIME25 maxed), so they can't reveal quantization degradation.
- No KLD (KL divergence) numbers are published, leaving distribution-level damage unmeasured.
The author asks whether anyone has run their own evaluations and what KLD/PPL would be expected at 2 bpw — a reminder that 'lossless 2-bit' claims need more sensitive metrics to verify.
More from Infra
- vLLM serves DeepSeek-V4.1-Flash at 954K tokens/s per GPU, matching DeepInfer at 7.9× throughput — Xianbao_QIAN · 2026-10-25
- Amazon's ATTUNE coding agent lifts LLM training throughput 1.7x on H100/H200/B200 clusters — kalyan_kpl · 2026-10-25
- HKUST & Huawei's BalanceRoute tackles load balancing in MoE LLM serving — jiqizhixin · 2026-10-25
- Pushing local AI video to the limit: MiniMax H3 running on a single RTX 3090 — DeliciousGorilla · 2026-10-25
- a16z: GPU rental rates defy depreciation, H200 up 143% YTD — elonmusk · 2026-10-25
- Qwen3.8 Flash Next IQ3_S runs locally on 16GB VRAM at 250 tok/s generation, 5,600 tok/s prefill — yehyakar · 2026-10-25