AWS Guide: Deploying 2.8T Parameter Kimi K3 Requires 8x B300 GPUs
AWS ML Blog · rss · 2026-07-31
AWS published a detailed guide for deploying the Kimi K3 model on its cloud infrastructure. Kimi K3 is a 2.8 trillion parameter Mixture of Experts (MoE) model with 104 billion active parameters per token and a 1 million token context window.
The post outlines two primary deployment paths:
- Amazon SageMaker HyperPod: Uses the Inference Operator to abstract away container orchestration and model loading, requiring Flexible Training Plans for capacity reservation.
- Amazon EKS: For teams managing their own Kubernetes clusters, utilizing EC2 Capacity Blocks to reserve instances.
Infrastructure Requirements: Deploying this model requires the ml.p6-b300.48xlarge instance equipped with 8 NVIDIA B300 GPUs. The recommended serving engine is a dedicated vLLM container, and model weights are distributed in MXFP4 format for memory efficiency.
More from Infra
- Trading 2 RTX 5090s for 4 R9700s: does doubled VRAM pay off? — poofph · 2026-10-16
- Running PolyStrata + Cline coding agent on a $0-fancy laptop at ~90 tok/s — rpwoerk · 2026-10-16
- TensorFold 1.0.12 runs 120B vision model on 1-2 DGX Sparks, adds Mac image input — HankYeomans · 2026-10-16
- vLLM brings frontier model local deployment to NVIDIA RTX Spark PCs — vllm_project · 2026-10-16
- NVIDIA signals $2B in new European AI deals at GTC Berlin, totaling ~$10B in a year — CurieuxExplorer · 2026-10-16
- Making Your Data Ready for Agentic AI: trust first, then semantics, then writes — bibryam · 2026-10-16