Kimi says MI355X serving changes cut p99 TTFT by 3.2× and lift throughput 7.7%
AnushElangovan · x · 2026-07-23
Kimi and AMD say they rebuilt the serving stack for Kimi K2.6 on AMD Instinct MI355X, targeting agent workloads with long contexts, bursty traffic, and frequent tool calls.
The update adds scheduler-aware multi-tier KV caching and claims up to 3.2× lower p99 TTFT plus 7.7% higher total-token throughput, with no accuracy loss. The post points readers to the technical blog for details.
More from Infra
- RTK Terminal Compression Cuts Tokens but Leaves Your AI Coding Bill Unchanged — Bartaseth · 2026-09-11
- SF Compute founder: buying compute is 'an absolutely awful experience' right now — IgorCarron · 2026-09-11
- SmolVM open-sources persistent computer infrastructure for agents that outlive chat sessions — aniketmaurya · 2026-09-11
- PyTorch Day Korea 2026 launches first offline conf, CFP closes Sept 13 — PyTorch · 2026-09-11
- Local LLM server dilemma: 4x CMP-170HX (price up 53% in 20 days) vs Mac Studio M5 Ultra — rumboll · 2026-09-11
- llama.cpp lands Flash Attention tuning for RDNA4, big prefill gains on AMD — pmttyji · 2026-09-11