AMD MI355X Runs Kimi K3 at 952 tok/s, Outperforming B200
The Wafer.ai engineering team successfully deployed the Kimi K3 model on a single AMD MI355X server. Tests show the setup achieved an aggregate throughput of 952 tok/s and a single-stream decoding speed of 118 tok/s, reducing single-node costs by 71% compared to NVIDIA's B200. This demonstrates MI355X's strong cost-effectiveness for large model inference due to its memory advantages.
Confirmed
- Kimi K3 has 2.8T parameters and requires over 1.5TB of VRAM, exceeding the capacity of a single B200 node, whereas the MI355X accommodated it on a single node.
- MI355X single-node test results: aggregate throughput reached 952 tok/s, with single-stream decoding at 118 tok/s.
- Comparison with B200: aggregate throughput increased by 3.8 times, single-stream decoding by 1.3 times; performance per dollar reached approximately 48 tok/s, with single-node costs reduced by 71%.
Unconfirmed
- These performance and cost figures originate solely from Wafer.ai's internal evaluations and lack independent third-party verification; long-term stability under real-world production loads, varying batch lengths, and high concurrency remains to be seen.
Why it matters
- Memory capacity has emerged as a critical advantage for AMD chips in this context, proving their strong competitiveness in deploying ultra-large models with massive VRAM demands.
- Being able to run a 2.8T model like Kimi K3 on a single node significantly lowers deployment barriers and computing costs for enterprises.
2026-08-02 ~ 2026-08-04 · 6 related posts
Primary sources
- AMD MI355X Runs Kimi K3 at 952 tok/s/node, Outperforming B200 — AnushElangovan ·
- AMD MI355X Hits 952 tok/s Serving Kimi K3, Beating Nvidia B200/B300 in Efficiency — AnushElangovan · 2026-08-02
- [source] AMD MI355X Runs Kimi K3 at 952 tok/s/node, Outperforming B200 — AnushElangovan · 2026-08-02
- Wafer serves Kimi K3 on AMD MI355X with 3.8x throughput and 71% lower cost vs B200 — SumitGup · 2026-08-03
- AMD MI355X Beats NVIDIA B200 in Kimi K3 Deployment with 952 tok/s — adrianscottcom · 2026-08-03
- Serving Kimi K3 on AMD MI355X: 3.8x Throughput and 71% Lower Cost vs B200 — SumitGup · 2026-08-04
- Kimi K3 runs on AMD MI355X at 952 tok/s per node, claim says — AccBalanced · 2026-08-04