Disaggregated Quantization lifts 1-bit Qwen3.8-27B from 29% to 61.5% on MMLU-Pro
udmrzn · x · 2026-10-14
A new paper from ISTA-DASLab uses Disaggregated Quantization to lift a 1-bit Qwen3.8-27B from 29.04% to 61.54% on MMLU-Pro — a 32.5-point gain.
Instead of swapping in a bigger decoder, the method splits responsibilities across two weight sets:
- Prefiller: a trained NVFP4 model processes the prompt and builds the KV cache, trained specifically for the compressed decoder
- Decoder: the heavily quantized 1-bit GGUF model generates the response
The same group previously produced the GSQ-RCO quantizations of Qwen3.8-27B.
More from Infra
- Same model, 9x cost gap: Artificial Analysis publishes cache hit rates per provider — ArtificialAnlys · 2026-10-14
- Apple's Minions turns idle Mac GPUs into a pooled cloud for ML inference — MikePFrank · 2026-10-14
- AnythingLLM ditches Ollama for llama.cpp, with up to 2x faster inference — solyarisoftware · 2026-10-14
- LlamaForge: a browser panel for upstream llama.cpp with local agents that must quote their sources — Sleepybear2611 · 2026-10-14
- CoreWeave's GPU fleet is mostly older boxes, raising questions about depreciation math — ivan_bezdomny · 2026-10-14
- NVMe reads the 7GB checkpoint in 2s, so why does ComfyUI take 30s to load? — rguerraf · 2026-10-14