Kimi K3 Enabled on AMD MI350 and NVIDIA B200 with Day-0 Inference Optimizations
zhyncs42 · x · 2026-07-28
The TokenSpeed team collaborated closely with Moonshot to achieve Day 0 support for the Kimi K3 model on NVIDIA Blackwell (B200/B300) and AMD Instinct (MI350X/MI355X) GPUs in just one week.
Core inference optimization techniques include:
- Prefix caching
- Speculative decoding
- Disaggregated serving
- CUDA Graph decode
The team also shared underlying optimization details for KDA, Gated MLA, Stable LatentMoE, AttnRes, and their unified Flat KV architecture.
Related event: TokenSpeed Enables Kimi K3 Support on NVIDIA and AMD Platforms(2 posts)→
More from Infra
- Musk: robot RAM demand will dwarf everything else, and 100% of superintelligence compute goes to deep space — elonmusk · 2026-10-27
- RTX 5090 hits $7K as PC gaming community goes into 'denial' over AI boom — PAstynome · 2026-10-27
- Wan2GP lands on Pinokio: 1-click AI video on 6GB GPUs — cocktailpeanut · 2026-10-27
- Wan2GP AMD one-click installer lands on Pinokio, runs AI video on 6GB VRAM — cocktailpeanut · 2026-10-27
- AWS on agent prompt caching: explicit + implicit patterns to slash token costs — AWS ML Blog · 2026-10-27
- Thieves spoofing GPS steal $100M+ in data center cargo since June — Ars Technica AI · 2026-10-27