SGLang says it can serve Kimi K3 at 423 tok/s with day-0 production support
vwxyzjn · x · 2026-07-28
SGLang says it can serve Kimi K3 at day-0 speed, measuring 423 tokens/s on GSM8K, and says RL support is already ready in Miles.
The post explains that SGLang natively implemented and optimized K3’s new architecture with fused KDA decode kernels, DP attention, DSpark, PD disaggregation, and KDA-aware prefix caching. It says the model passed Kimi Vendor Verifier and is ready for production, with benchmarks, a blog post, and a cookbook linked in the comments.
Related event: SGLang Day-0 Support for Kimi K3 Boosts Throughput to 423 tok/s(8 posts)→
More from Infra
- Encrypted GPT-2 speedrun challenges FHE inference: best is 3.031 s/token, goal sub-1s — tensorqt · 2026-10-27
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27
- NVIDIA claims HGX Rubin NVL8 delivers 10x tokens per megawatt vs B200 — thursdai_pod · 2026-10-27
- NVIDIA Claims HGX Rubin NVL8 Delivers 10x More Tokens per Megawatt Than B200 — thursdai_pod · 2026-10-27
- Together lets multiple LoRA post-training experiments share the same GPU capacity — togethercompute · 2026-10-27
- Commoditization dilemma: either human labor or GPUs become the commodity — _AustinCalvert_ · 2026-10-27