SGLang Partners with Google and RadixArk to Deliver Native TPU Inference for LLMs
ying11231 · x · 2026-07-31
Open-source LLM inference framework SGLang announced a collaboration with Google Cloud and RadixArk to optimize inference for large language and diffusion models on TPUs.
Currently, the JAX-based SGL-JAX offers a production-level solution, delivering fast, native TPU inference for major model families like Gemma, Qwen, and DeepSeek. This partnership will enable TPUs to unlock SGLang's full production feature set, including 5D parallelism, Radix Cache, quantization, and speculative decoding. The team is also developing SGL-torchtpu to bring a PyTorch-native path to TPU inference.
Related event: SGLang Joins Google Cloud and RadixArk to Bring TPU Inference(6 posts)→
More from Infra
- Meta's commitments jumped to $349B in a quarter as server memory prices tripled ahead of Oct 21 earnings — tengyanAI · 2026-10-16
- Meta releases Muse Spark Open, trained with vLLM-powered RL inference from day one — colinmcnamara · 2026-10-16
- Meta's open Muse Spark needs 493GB of memory — far beyond any consumer laptop — TejasKumar_ · 2026-10-16
- CoreWeave plunges 20% as credit markets price >60% default odds; is this dot-com all over again? — kimmonismus · 2026-10-16
- MiniCPM-V 4.7 35B-A3B hits 17+ tokens/s on iPad with ~3GB memory — techNmak · 2026-10-16
- China rolls out 'token factories' and a 2026-28 token economy action plan — nordicinst · 2026-10-16