DeepSeek-V4.1-Flash quantized to 3-bit for 3x RTX PRO 6000, KLD only 0.066 nats
TheZachMueller · x · 2026-10-19
BrandonMusicKy released a 3-bit TrellisMX quantization of DeepSeek-V4.1-Flash-1017, laid out for three RTX PRO 6000 GPUs with Engram tables at fp8 ngram.
- Mean tokenwise KL divergence vs the official checkpoint: 0.0664 nats on 2,048-token sequences, with 91.7% top-1 agreement
- KLD rises to 0.130 on longer 8,192-token windows
- The build and all KL measurements ran on an 8x B200 node donated by Lambda
- With enough RAM, two spark instances can run simultaneously
The model is up on Hugging Face (brandonmusic/DeepSeek-V4.1-Flash-1017-TrellisMX-K3).
More from Infra
- CMU seminar digs into Floe lakehouse engine: no buffer pool, I/O faster than io_uring — sh_reya · 2026-10-19
- Data centers use just 0.85% of global primary energy, by the numbers — DarinFeinstein · 2026-10-19
- vLLM 0.32.0 adds structured-decision API reading first-token probabilities — AccBalanced · 2026-10-19
- GPU pricing is now set by cost of capital, not spot rates, argues Maya — AccBalanced · 2026-10-19
- Liquid Inference ships DeepSeek V4.1-Flash support, ComfyUI plugin over weekend — AccBalanced · 2026-10-19
- Analyst revises 2027 datacenter forecast up to +50GW, citing satellite imagery — AccBalanced · 2026-10-19