Google DeepMind's free 'How To Scale Your Model' book demystifies LLM systems
mdancho84 · x · 2026-11-05
Matt Dancho highlights Google DeepMind's free online book How To Scale Your Model, a systems view of LLMs on TPUs and GPUs by Jacob Austin, Sholto Douglas, Roy Frostig and other DeepMind researchers (published Feb 2025).
- Covers how accelerators work and communicate, how LLMs run on real hardware, and how to parallelize training and inference
- Answers practical questions: how much an LLM should cost to train, how much memory you need to serve a model, what an AllGather is, and how to choose parallelism schemes
- Requires basic LLM/Transformer knowledge, ideally some JAX familiarity
The thread also points to Part 12, "How To Think About GPUs," with annotated diagrams of SMs, Tensor Cores, warp schedulers, and the L1/L2 cache/HBM memory hierarchy, including concrete H100/B100 specs.
Related event: Google DeepMind Releases Free 'How To Scale Your Model' Textbook(2 posts)→
More from Infra
- LFM3-Omni-2B ported to run fully offline on iPhone at ~80 tok/s with custom WebGPU kernels — JosephJacks_ · 2026-11-05
- Together Compiler (TKCC) uses agents to auto-optimize inference, 252x more experiments — togethercompute · 2026-11-05
- Orbital Data Centers Launch to Sun-Synchronous Orbit, Memed as Elves Sailing West — rand_longevity · 2026-11-05
- Vietnam's Lâm Đồng to bring Starlink to mountain areas where poor signal hampers rescue — elonmusk · 2026-11-05
- SpaceX partners with Qualcomm for nationwide Starlink Mobile 5G network from 2027 — Polymarket · 2026-11-05
- Stopping runaway agent spend before the bill: pre-call guardrails and three gaps found the hard way — MaverikSh · 2026-11-05