Dev ships ik_llama.cpp quants for DeepSeek V4 Flash, beats Atomic in paired KLD tests
KeinNiemand · reddit · 2026-08-28
KeinNiemand published a full ikllama.cpp GGUF ladder for DeepSeek V4 Flash 0731 on Hugging Face, from a 65.1 GB XSIQ1KT up to a 149.1 GB IQ4KSS.
Quant details
- Built with an importance matrix; routed MXFP4 tensors requantized from native representation, FP8 tensors expanded to BF16 first.
- Requires a recent ikllama.cpp build — mainline llama.cpp won't load the IK-specific IQK/IQT tensor types.
Paired quality testing
Using AtomicChat's published lossless BF16 reference logits and WikiText-2 tokens under an identical local setup, the author compared PPL, mean KLD, RMS delta-p and top-1 agreement:
- 93 GB: IQ2KS vs Atomic AD-IQ2S: 8.35% lower mean KLD, 0.100 lower PPL, +0.92pp top-1 agreement.
- 73 GB: IQ1KT shows 19.03% lower mean KLD than Atomic AD-IQ1MXL.
Takeaways
- IQ1KT and IQ2KS are probably among the best quality-per-size options at their sizes, though the author only tested two comparison points and excluded two Atomic runs with invalid results; the three largest quants could not be KLD-tested due to memory limits.
- Don't treat GGUF file size as total memory need — budget for KV cache, runtime buffers and non-offloaded tensors. IQ2KS is the default pick at 93 GB; IQ2KT is similar quality but slower on CPU/hybrid inference.
More from Infra
- SK Hynix ties new US plant to looser chip equipment exports to China, sources say — pstAsiatech · 2026-10-15
- Perplexity's new embedding models: index with a 9B model, search with 0.6B — antoine_chaffin · 2026-10-15
- Unsloth ships GGUF quants for Nemotron 3.5: Bolt-74B runs in 24GB RAM — danielhanchen · 2026-10-15
- vLLM-Omni ships Day-0 serving for FLUX 3 Image Klein with BF16, FP8, NVFP4 support — vllm_project · 2026-10-15
- NVIDIA's Nemotron 3.5 Super VL lands training and inference support on Applied Compute — ypatil125 · 2026-10-15
- CoreWeave bond yield tops 16% as loans outlast customer contracts by ~2 years — tengyanAI · 2026-10-15