Kimi says MI355X serving changes cut p99 TTFT by 3.2× and lift throughput 7.7%
AnushElangovan · x · 2026-07-23
Kimi and AMD say they rebuilt the serving stack for Kimi K2.6 on AMD Instinct MI355X, targeting agent workloads with long contexts, bursty traffic, and frequent tool calls.
The update adds scheduler-aware multi-tier KV caching and claims up to 3.2× lower p99 TTFT plus 7.7% higher total-token throughput, with no accuracy loss. The post points readers to the technical blog for details.
More from Infra
- Anthropic S-1 chart fuels positive-ROI narrative on AI infrastructure buildout — BenBajarin · 2026-10-26
- Nearly half of Claude revenue now flows through cloud marketplace commitments, up from 11% — BenBajarin · 2026-10-26
- Anthropic ARR goes vertical after price cuts: tokens up 4x, $100B+ ARR back in play — GavinSBaker · 2026-10-26
- Alex Banks argues power caps GPU supply, real risk is demand cooling first — thealexbanks · 2026-10-26
- Analyst: Ant has the most fungible AI workload of any model provider today — BenBajarin · 2026-10-26
- Anthropic S-1: training infra edge cut per-task cost 40% and doubled revenue growth — BenBajarin · 2026-10-26