Google open-sources TPU codesign guide: 8 principles for LLM/MoE performance
rseroter · x · 2026-11-19
Google launched the open-source AI-Hypercomputer/tpu-codesign guide on its developers blog, covering how to design and optimize models for LLMs, MoE, and other compute-heavy workloads on TPUs (targeting Ironwood/TPU v7x and beyond). Key thesis: architectural choices made before the first training step set permanent ceilings on MFU, cost, and serving latency. The guide groups rules into three tiers:
- Shape rules (Principles 1–2): hardware-aligned dimensions and static shapes; near-zero quality risk, adopt always.
- Architecture choices (Principles 3–7): structural decisions like LatentMoE; most involve quality tradeoffs, validate with ablations.
- Expressing parallelism (Principle 8): parallelism, kernel, and compiler tuning with no model changes.
New model designs should start with shape rules; adapting existing models mainly needs the parallelism section.
More from Infra
- vLLM team's Inferact to build an autonomous inference system around vLLM — vllm_project · 2026-11-19
- AI Stocks Plummet as Burry's Bubble Call Lands: CoreWeave -74%, Oracle -65% — SumitGup · 2026-11-19
- Banks offload $800M of AI data center debt, including loans tied to Oracle's Stargate — SumitGup · 2026-11-19
- BIS interim rule targets cloud remote access to US chips; CNAS unpacks what it gets right and wrong — paul_scharre · 2026-11-19
- Japanese neocloud Datasection slashes revenue guidance 89%, stock limit-down — dylan522p · 2026-11-19
- Matt Wolfe's video essay: the reason no one can catch NVIDIA — Matt Wolfe · 2026-11-19