Kimi K3 adds native MXFP4 quantization and shows heavy VRAM demands
teortaxesTex · x · 2026-07-27
- The image excerpt says Kimi K3 uses native MXFP4 quantization: MXFP4 weights with MXFP8 activations.
- It claims quantization-aware training starts from the SFT stage to improve hardware compatibility.
- The deployment section says Kimi K3 is accessible through the Kimi platform API and offers OpenAI/Anthropic-compatible APIs.
- It recommends running inference on vLLM, SGLang, or TokenSpeed.
- The accompanying hardware slide says the model’s VRAM needs are large because its MoE experts must be loaded in memory, and gives rough totals such as 480 GB minimum VRAM and 8× H100 as a recommended configuration in one setup.
Related event: Moonshot releases Kimi K3 open weights amid license debate(155 posts)→
More from Infra
- AWS tutorial: fleet-wide edge CV model lifecycle with SageMaker SDK v3, ~$0.07 per run — AWS ML Blog · 2026-10-27
- Qwen3.8-27B hits 40 tok/s on Jetson Thor with INT4 + MTP, accuracy beats FP16 — No-Dragonfly6246 · 2026-10-27
- Celesto launches CI runners nearly 90% cheaper than GitHub Actions, with agent sandboxes — aniketmaurya · 2026-10-27
- Supabase founders on rebuilding the backend for a billion AI-built apps — Latent Space · 2026-10-27
- AW-0 runs at 230ms on a single RTX 5090, 2.4x faster than autoregressive code planning — trevordarrell · 2026-10-27
- Site tracks US city council topics: data centers rank only behind pickleball courts — zakkohane · 2026-10-27