Red Hat Releases Quantized Kimi K3: FP4/FP8 for Accelerated Inference with Minimal Quality Loss
_akhaliq · x · 2026-08-01
Red Hat AI has released hardware-optimized quantized versions of Moonshot's Kimi K3 model to accelerate local inference and boost throughput.
- NVFP4 Version: Designed for Blackwell architecture, quantizing MoE layers to 4-bit. Evaluations show minimal quality degradation, with GPQA score dropping slightly from 93.5 to 91.0.
- FP8-Block Version: Optimized for Hopper architecture (H100/H200), leveraging native Tensor Cores for high throughput with Day Zero support via vLLM.
The quantized weights are now open-sourced on Hugging Face and recommended for deployment with vLLM.
More from Infra
- China's AI boom drives up to 660% profit surges for domestic chip firms; GPU maker MetaX turns profitable — pstAsiatech · 2026-10-16
- Qdrant tests Perplexity's visual PDF embeddings: 21x smaller vectors, 99.9% after rescoring — jrxna · 2026-10-16
- When Does an Agent Actually Need an API Gateway? Reddit Weighs Direct Calls vs. Unified Endpoints — TurbulentStiffness · 2026-10-16
- Taiwan to build $4B AI manufacturing complex in Texas to back Nvidia and TSMC — Polymarket · 2026-10-16
- When the KV Cache Is Slower Than the GPU: Remote KV Tiers Backfire in Production — MultiverseComputingCAI · 2026-10-16
- GitHub rewrote the 800k-line Copilot runtime in Rust using AI agents, cutting CPU and memory by over 90% — bibryam · 2026-10-16