Huawei open-sources MACaQ quantizer: 23% lower KL than Unsloth GGUFs
acluk90 · reddit · 2026-11-12
Huawei's CSL team open-sourced MACaQ (Model Agnostic Calibrated Quantization), a quantizer supporting text/image/audio/video/3D and compatible with llama.cpp, vLLM, and Diffusers.
Benchmarks (IQ4XS GGUFs, near-identical file sizes):
- Qwen3-4B: mean KL divergence 23.5% lower than Unsloth, 18.6% lower than Bartowski
- Qwen3.8-27B: 65% lower KL than Bartowski (0.0368 vs 0.1042)
- Supports IQ2/IQ3/IQ4 formats across the llama.cpp ecosystem
- Huawei also reports quantizing a 122B model in 64 minutes with just 32.5GB of GPU memory
Code, llama.cpp fork, and benchmarks are on GitHub.
Related event: Huawei Open-Sources MACaQ, a Model-Agnostic Calibrated Quantization Method(2 posts)→
More from Infra
- Data centers power 20% of US tech sector and half of decade's stock gains — SydSteyerhart · 2026-11-12
- Budget GPU server rental tips: OVH KS-6/7, FiberState, and asking for unlisted stock — realmrfakename · 2026-11-12
- AMD EPYC review image looks uncannily like logit artifacts from ViT segmentation models — giffmana · 2026-11-12
- llama-modchip runs 284B DeepSeek-V4-Flash IQ1 on a pure 64GB CPU with zero GPUs — pmttyji · 2026-11-12
- Agent Substrate Calculator Estimates Session Resume Speed Under Load — pjausovec · 2026-11-12
- Dell expands AI Data Platform: agents have a data-context problem, not a model problem — DavidLinthicum · 2026-11-12