SGLang Comes to Google TPUs: Enabling Seamless Migration for Major LLMs
BanghuaZ · x · 2026-07-31
Google Cloud and RadixArk have partnered to bring the open-source inference framework SGLang to TPUs.
- Current Support: Developers can run SGLang on the latest TPU generations via SGL-JAX, supporting major LLMs like Gemma, Qwen, DeepSeek, Kimi, and Grok, alongside diffusion models like Wan and Flux.
- Future Roadmap: Later this year, RadixArk will roll out SGL-torchtpu, a PyTorch-native backend featuring full parallelism capabilities (data, tensor, expert, context, pipeline) for full-size frontier models on multi-host TPUs.
- Ecosystem Goal: Pledges Day 0 support for new open models on TPU, making TPUs a drop-in, cost-efficient path for frontier inference.
Related event: SGLang Joins Google Cloud and RadixArk to Bring TPU Inference(6 posts)→
More from Infra
- Trading 2 RTX 5090s for 4 R9700s: does doubled VRAM pay off? — poofph · 2026-10-16
- Running PolyStrata + Cline coding agent on a $0-fancy laptop at ~90 tok/s — rpwoerk · 2026-10-16
- TensorFold 1.0.12 runs 120B vision model on 1-2 DGX Sparks, adds Mac image input — HankYeomans · 2026-10-16
- vLLM brings frontier model local deployment to NVIDIA RTX Spark PCs — vllm_project · 2026-10-16
- NVIDIA signals $2B in new European AI deals at GTC Berlin, totaling ~$10B in a year — CurieuxExplorer · 2026-10-16
- Making Your Data Ready for Agentic AI: trust first, then semantics, then writes — bibryam · 2026-10-16