Kimi K3 hits 391 tokens/s per user on TPU v7 with open-source vLLM megakernels
woosuk_k · x · 2026-10-21
Inferact reports Kimi K3 running on vLLM TorchTPU reaching 391 tokens/s per user on SemiAnalysis' AgentX benchmark, powered by their TPU megakernels.
The result holds up on real, long-context, production-level agentic workloads. It's publicly viewable on InferenceX, and the megakernels are open source.
More from Infra
- Anthropic's draft IPO filing: $16.3B H1 revenue, $80B net loss, $750B compute plans — kimmonismus · 2026-10-21
- dphnAI's distributed inference network served 32B DeepSeek tokens in 24h, ~6% of Venice's daily volume — Ronangmi · 2026-10-21
- Jensen Huang: every company will have two factories, one real and one digital — round · 2026-10-21
- DeepSeek V4 called 'god-tier infra, 4th-tier ML': 7 tricks stacked, 20 patches to fix them — teortaxesTex · 2026-10-21
- HSBC says CoreWeave shares to drop 30%+ as ROIC of 8.9% lags borrowing costs — GaryMarcus · 2026-10-21
- RunInfra launches $10/mo coding plan: fastest open models across 19 agents incl. Claude Code — ycombinator · 2026-10-21