Kimi K3.1 Pro hits 300 tok/s on 8x MI350X, a 5x speedup over K3
AnushElangovan · x · 2026-11-07
tinygrad ran Kimi K3.1 Pro Day 0 on 8x AMD MI350X with the default SGLang recipe, hitting 300 tok/s decode — up from 60 tok/s with K3 — making it often a better pick than Astra/Opus.
Benchmark table:
- 127-token prompt: TTFT 0.892s, 295 tok/s decode
- 996 tokens: TTFT 0.827s, 1,205 tok/s prefill, 291 tok/s decode
- 4,045 tokens: TTFT 1.224s, 3,293 tok/s prefill, 308 tok/s decode
- 16,125 tokens: TTFT 2.706s, 5,993 tok/s prefill, 310 tok/s decode
More from Infra
- HyperFlow H3 Flash: 13.7B Few-Step Model Generates Video With Stereo Audio, Runs on RTX 5090 — Apprehensive_Sky892 · 2026-11-07
- JITEngine synthesizes custom LLM inference engines for Apple Silicon, claims up to 2.7x speedup — AccBalanced · 2026-11-07
- Dev predicts 2027's local inference boom: integrated hardware + OS + inference stack, not just GPUs — vincent_koc · 2026-11-07
- vLLM Office Hours Hits 59 Sessions: 76 Speakers from 24 Organizations and Counting — vllm_project · 2026-11-07
- Dev runs 284B-parameter DeepSeek-V4-Flash on CPU only, squeezed into 64GB DDR5 RAM — solyarisoftware · 2026-11-07
- Omnigraph: open-source graph database with git-style branching for multi-agent shared memory — solyarisoftware · 2026-11-07