2-bit quantized DeepSeek 4.1 Flash tested locally: KDL degrades but still works
bakawolf123 · reddit · 2026-10-23
Reddit user bakawolf123 benchmarked a local 2-bit quant of ds4.1-flash-1017 against cloud DeepSeek APIs, inspired by an X post comparing opencode with ds4.1 via the official API. The run took 1.5h at 450k context (auto-compaction on) and validated on headless Chrome since Playwright wasn't available. Measured quality: KDL vs baseline of 0.4259 nats (oQ2e) and 0.2821 (oQ2.5e); perplexity rising from 3.7315 (original) to 4.1190/4.6333; top-1 agreement 78.43% for the 2-bit model. Notably, on glm5.3-flash a 0.12 KDL gap between calibrated and non-calibrated quants was clearly visible in quality, so the author finds it surprising this 2-bit version still works as well as it does.
More from Infra
- Virtual Head Attention: 4x smaller KV cache, beats GQA on long context (RULER 84.4 vs 73.0) — pmttyji · 2026-10-23
- NVIDIA ships Nemotron 3.5 Bolt E4B, an open vision model that runs on 6GB RAM — danielhanchen · 2026-10-23
- TensorSharp hits 4.4–5.5x llama.cpp prefill throughput on DeepSeek V4.1 Flash with 5x A40 — fuzhongkai · 2026-10-23
- SauerkrautLM-MoEdernColBERT: tiny 1.5M-115M MoE late-interaction retrievers, Apache 2.0 — QuixiAI · 2026-10-23
- Amazon CEO Andy Jassy: $220B capex still can't meet AI demand, AWS eyes $1T run-rate — a16z · 2026-10-23
- Developers now deploying 1T-parameter models on their own infrastructure — dejavucoder · 2026-10-23