Dev ships ik_llama.cpp quants for DeepSeek V4 Flash, beats Atomic in paired KLD tests
KeinNiemand · reddit · 2026-08-28
KeinNiemand published a full ikllama.cpp GGUF ladder for DeepSeek V4 Flash 0731 on Hugging Face, from a 65.1 GB XSIQ1KT up to a 149.1 GB IQ4KSS.
Quant details
- Built with an importance matrix; routed MXFP4 tensors requantized from native representation, FP8 tensors expanded to BF16 first.
- Requires a recent ikllama.cpp build — mainline llama.cpp won't load the IK-specific IQK/IQT tensor types.
Paired quality testing
Using AtomicChat's published lossless BF16 reference logits and WikiText-2 tokens under an identical local setup, the author compared PPL, mean KLD, RMS delta-p and top-1 agreement:
- 93 GB: IQ2KS vs Atomic AD-IQ2S: 8.35% lower mean KLD, 0.100 lower PPL, +0.92pp top-1 agreement.
- 73 GB: IQ1KT shows 19.03% lower mean KLD than Atomic AD-IQ1MXL.
Takeaways
- IQ1KT and IQ2KS are probably among the best quality-per-size options at their sizes, though the author only tested two comparison points and excluded two Atomic runs with invalid results; the three largest quants could not be KLD-tested due to memory limits.
- Don't treat GGUF file size as total memory need — budget for KV cache, runtime buffers and non-offloaded tensors. IQ2KS is the default pick at 93 GB; IQ2KT is similar quality but slower on CPU/hybrid inference.
More from Infra
- Three 32GB AMD R9700s for the price of one RTX 5090: a local LLM user's dilemma — MrHall · 2026-08-28
- Vulkan runs 20°C cooler than CUDA on laptops in llama.cpp, with a catch — Hot-Employ-3399 · 2026-08-28
- DeepSeek V4 Flash pricing puzzle: How does it match GPT OSS 20B? — gajesh · 2026-08-28
- Qwen3.8-Flash reportedly costs 1/9th to train vs Qwen3.7-Plus — VraserX · 2026-08-28
- Sovereign AI market hits $1.5T as companies flee US cloud providers — mikeflache · 2026-08-28
- ComfyUI multi-GPU setups: text+VAE on one card, diffusion on the other — hurdurdur7 · 2026-08-28