SGLang-TorchTPU launches with Google and Meta, hitting 770 tok/s decode on TPU
BanghuaZ · x · 2026-10-22
SGLang announced SGLang-TorchTPU, built jointly with Google's TPU team and Meta's PyTorch team, already powering production inference at Meta.
- A TPU plugin for SGLang that runs top open models on Ironwood TPUs: Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, and MiniMax H3
- Brings SGLang's production features to large agentic workloads
- Custom MegaKernel: Muse Spark Open reaches 770 tok/s decode at bs=1 on InferenceX 8K/1K
- The team notes the TPU ecosystem has grown rapidly since Ironwood's announcement
A significant step for the TPU inference stack and open-model deployment.
More from coding & agent
- When AI automation saves less than it costs: the hidden maintenance tax of agents — djgoel123 · 2026-10-22
- PHP ships emergency fixes for two unauthenticated RCE bugs after embargo break — kingslyj · 2026-10-22
- Flying a drone with Codex: 12–20s per decision is the bottleneck — kfountou · 2026-10-22
- One week of vibe coding, 30,708 players in month one: Dust 2 DM game's Spawn stats — TAbrodi · 2026-10-22
- Security patch ships with 1,858× speedup: 3,000-scene queries drop from 2s to 1.13ms — DanielLockyer · 2026-10-22
- Codiff v1.18 ships faster startup, prettier markdown diffs and a new app icon — cnakazawa · 2026-10-22