Kimi K3 may need 1.4 TB for MXFP4 inference, far beyond a 96 GB RTX PRO 6000
HanchungLee · x · 2026-07-28
A post argues that Kimi K3 would need about 1.4 TB of memory for MXFP4 inference, while an RTX PRO 6000 has only 96 GB GDDR. The claim implies roughly a 93% reduction in model size would be needed to run it on a single card without losing capability.
The author says distillation could compress it further — similar to R1 → Qwen32B — but warns that knowledge loss would likely make the result “jagged.” The quoted prediction being debated is that a K3-level system could run on one RTX PRO 6000 within 18 months.
Related event: Moonshot Kimi K3 Poses Extreme Local Deployment Barriers(2 posts)→
More from Infra
- Mapbox opens Places API with metadata for 250M places worldwide — anselm · 2026-07-28
- Inference.net pitches a gateway flow that mirrors prod traffic to Kimi K3 before switching — MatthewBerman · 2026-07-28
- AI is likely to control quantum computers first, then use them for narrow science tasks — imjustnewatai · 2026-07-28
- Moonshot’s Kimi K3 lands in Japan with 2.8T open weights and $3/$13 pricing — DavidBennett__ · 2026-07-28
- A Kimi-k3 joke contrasts a $1,908 annual plan with $1.0908M to run it at home — HarveenChadha · 2026-07-28
- Calibrated Qwen3.6-27B quantization tests weight groups before compressing them — enginetown · 2026-07-28