oQe Quantization Update Cuts Code KL Divergence by Up to 18% Across 69 Models
huggingface · x · 2026-10-13
oMLX creator Jun Kim shipped improved oQe quantized models with oMLX 0.7.1, updating 69 models including Qwen3.8-27B and Qwen3.8-Flash-Next.
- KL divergence vs BF16 drops 11.6–18.1% on code and 10.2–13.8% on prose (e.g. Qwen3.8-27B oQ4e code: 0.0143 → 0.0117)
- Below oQ4, Engram and N-gram tables stay at 4 bits, excluded from the bit budget, and can be served from SSD instead of RAM
- Model cards now show bit distribution and storage breakdown
Earlier oQe builds should be re-downloaded. Kim recently joined Hugging Face to support the MLX community.
More from Infra
- The Neocloud Map: physical ownership vs platform depth are separate axes — AccBalanced · 2026-10-13
- Will compute trade like oil? GPU rental indexes show B300 down 8.5% — AccBalanced · 2026-10-13
- Ai2 ships Olmo-core 3 training framework scaling MoE LLMs past 1 trillion parameters — dl_weekly · 2026-10-13
- Liquid Inference serves ~2B tokens in first days, claims cheaper prices than 98% of OpenRouter traffic — AccBalanced · 2026-10-13
- Solo dev turns old laptops into free security guards with local LLMs via Observer — Roy3838 · 2026-10-13
- NVIDIA livestream: memory optimization tricks to run bigger AI models on cheaper Jetson hardware — NVIDIA Developer · 2026-10-13