MagicQuant Releases Qwen3.8 27B Hybrid Quantized GGUF Models
crossivejoker · reddit · 2026-08-21
The author released Qwen3.8 27B GGUF models generated via the MagicQuant hybrid discovery system, utilizing Unsloth dynamic v3 and Imatrix. The post provides a detailed comparison between MagicQuant hybrid quantizations and original Unsloth versions in terms of KLD (divergence) and model size. Data shows that MagicQuant's hybrids (e.g., MQ-Q6K1) maintain extremely low KLD (0.000703) while optimizing volume. MagicQuant is a benchmark-driven GGUF evaluation and hybrid discovery system that finds high-performance hybrid quantization schemes by learning tensor configurations and running tests.
More from Infra
- DSCO Router Launches Unified Gateway for Multi-Model Routing with BYOK Support — arthurcolle · 2026-08-24
- Open Source RobotSoul: Persistent Identity for Agents After Context Resets — robauto-dot-ai · 2026-08-24
- Offloading MoE models to RAM causes slow prefill speeds — former_farmer · 2026-08-24
- Etched Raises $1B Led by Jane Street to Validate Architecture-Agnostic AI Chips — TheTuringPost · 2026-08-24
- ConvRot Quant joins llama-cpp: Q6 accuracy nears Q8 quality — giveen · 2026-08-24
- LifeOS: A Local, Voice-Driven Personal Organizer — Extension-Bid-639 · 2026-08-24