Unsloth ships DeepSeek V4.1-Flash GGUFs: 1-bit cuts 508GB to 250GB keeping 88.2% accuracy

danielhanchen · x · 2026-10-20

Unsloth's Daniel Han announced a llama.cpp implementation and dynamic GGUFs for DeepSeek V4.1-Flash-1017. Key points:

• 1-bit quantization shrinks the model from 508GB to 250GB (-52%) while retaining 88.2% accuracy; runs on 128GB+ RAM devices.
• The engram/PLE module can be swapped between MXFP8, Q6K, Q5K, and IQ4NL; Q6K ≈ MXFP8 quality, IQ4NL loses a bit.
• The engram can stream fully from disk.
• The team calls V4.1 Flash 1017 the strongest open model to date; guide and GGUF downloads are available.

Original post →

More from Infra

• Report: Liang Wenfeng planned independent DeepSeek compute chips before Huawei persuaded him otherwise — teortaxesTex · 2026-10-20
• TwelveLabs Pegasus 1.5 lands on Amazon Bedrock: turn 2-hour videos into timestamped JSON — AWS ML Blog · 2026-10-20
• NVIDIA's Ian Buck explains how AI factories can flex power use to relieve the grid — nvidia · 2026-10-20
• p99 container startup latency cut 63%, saving ~19 hours of compute daily on Cloud Run — DanielLockyer · 2026-10-20
• Late Interaction in 2026: Experiment Shows Single-Vector Search Ranks Wrong Answers Higher — IgorCarron · 2026-10-20
• AMD, NVIDIA, and Google All Ship Major vLLM Inference Optimizations in Three Days — PyTorch · 2026-10-20

Infra channel →