vLLM TorchTPU launches: vLLM backend for Google Cloud TPUs with free capacity pool
Xianbao_QIAN · x · 2026-10-21
vLLM TorchTPU, a new vLLM backend for Google Cloud TPUs, has been released in collaboration with Google. The author, who has worked with Google's internal TPU stack since the TensorFlow 1.0 era, frames it as making TPU as accessible as GPUs.
Key capabilities: models already served via vLLM can now run on TPUs; open-source Pallas kernels; multi-host parallelism; KV cache offloading; and disaggregated serving. A free TPU capacity pool is being opened for community contributors.
More from Infra
- Corporate token spending growth collapses to 0.2%, Apollo warns compute demand may not be unlimited — SumitGup · 2026-10-21
- Kyutai shrinks Pocket TTS to 1.9M params, running 103x real-time on one CPU core — alexcovo_eth · 2026-10-21
- Neoclouds now gross ~$38 per GPU-hour at 155 trillion tokens/day per cluster — basedjensen · 2026-10-21
- DeepSeek reportedly earns $38 per GPU-hour on tokens — what does it mean for GPU pricing? — andrew_n_carr · 2026-10-21
- Hyperscaler insider: GPU order-to-install lead time has always been 1–2 quarters — BenBajarin · 2026-10-21
- Efficient Computer's Electron E1 chip claims 100x energy efficiency, named a Time Best Invention — rebeccakaden · 2026-10-21