Moonshot Releases 2.8T-Parameter Kimi K3; Modal Achieves 460 TPS with Speculative Decoding
sarahcat21 · x · 2026-07-30
Moonshot released Kimi K3, its latest open-weight multimodal model, now available on Modal. K3 features 2.8 trillion parameters, a 1M token context window, and native vision, ranking as the top open model on public intelligence indexes.
Architecture & Performance
K3 uses a Mixture-of-Experts architecture with 16 experts active per token. It introduces Kimi Delta Attention and Attention Residuals, delivering roughly 2.5x the scaling efficiency of K2, specifically built for long-horizon agentic tasks.
Inference Optimization
Modal provided Day 0 vLLM support and custom-trained a DFlash speculator tuned to K3's architecture, achieving 460 tokens/s. Users can access it via the Shared API with token-based pricing.
More from Infra
- Trading 2 RTX 5090s for 4 R9700s: does doubled VRAM pay off? — poofph · 2026-10-16
- Running PolyStrata + Cline coding agent on a $0-fancy laptop at ~90 tok/s — rpwoerk · 2026-10-16
- TensorFold 1.0.12 runs 120B vision model on 1-2 DGX Sparks, adds Mac image input — HankYeomans · 2026-10-16
- vLLM brings frontier model local deployment to NVIDIA RTX Spark PCs — vllm_project · 2026-10-16
- NVIDIA signals $2B in new European AI deals at GTC Berlin, totaling ~$10B in a year — CurieuxExplorer · 2026-10-16
- Making Your Data Ready for Agentic AI: trust first, then semantics, then writes — bibryam · 2026-10-16