DeepSeek-V4.1-Flash quantized to 2-bit runs on two RTX PRO 6000s, KL eval published
TheZachMueller · x · 2026-10-19
- Release: A 2-bit TrellisMX quantization of DeepSeek-V4.1-Flash-1017 is now on Hugging Face, laid out for two RTX PRO 6000 GPUs, with Engram tables kept at 4-bit in 128GB of system RAM to cut VRAM requirements.
- Fidelity eval: Built and benchmarked on an 8x B200 node donated by Lambda. Against the official checkpoint with a full-vocabulary fp64 estimator: mean tokenwise KL divergence of 0.144 nats and 86.8% top-1 agreement on 2,048-token sequences; KL rises to 0.218 nats on 8,192-token windows.
- Reproducibility: Eval sets, metrics, and configs are published in the model repo.
More from Infra
- AI Capacity Isn't Sold Out — the Biggest Deals Are — DavidLinthicum · 2026-10-19
- Cobalt: The Ad-Free Downloader You Can Self-Host in 10 Minutes with Docker — JafarNajafov · 2026-10-19
- DFlash2 authors ship Qwen3.8-Flash speculative decoding model, 20% faster than MTP at concurrency 1 — rerri · 2026-10-19
- DOE lends $4.4B to restart three nuclear plants to feed AI data centers — emmanuelvivier · 2026-10-19
- China proposes export controls on 23 AI technologies, banning 2T-parameter models — emmanuelvivier · 2026-10-19
- Alibaba Cloud shows Omarchy running on Wuying AI cloud PCs for enterprise agents — AIFlow_ML · 2026-10-19