Google's free GPU masterclass breaks down Tensor Cores, caches and HBM for LLMs
mdancho84 · x · 2026-11-05
Part 12 of Google DeepMind's free book How To Scale Your Model, "How To Think About GPUs," is a deep dive into how NVIDIA GPUs work and what that means for LLMs, compared against TPUs.
- Core idea: a modern ML GPU (H100/B200) is essentially a stack of matrix-multiplication cores (Streaming Multiprocessors) attached to fast HBM memory
- Annotated diagrams cover Tensor Cores (the FLOPs workhorse, akin to TPU MXU), warp schedulers (SIMD units like TPU VPUs), SMEM/L1 cache (256kB on H100), L2 cache (50MB), and HBM (80GB on H100, 192GB on B100)
- Builds on Chapters 2 and 5 of the same book
Related event: Google DeepMind Releases Free 'How To Scale Your Model' Textbook(2 posts)→
More from Infra
- Unsloth Dynamic v3.1 GGUFs boost Qwen3.8-Flash-Next accuracy 10-15% over GSQ-RCO — danielhanchen · 2026-11-05
- Trained an MoE across two GPU clusters 25 Gbps apart — the compute islands guide — _arohan_ · 2026-11-05
- Arena data: Gemini 4 Carbon scales cheapest on hard tasks, rivals cost 3-4x more — arena · 2026-11-05
- Apple warns holiday-quarter gross margin will slide to 45% as memory chip prices surge — rohanpaul_ai · 2026-11-05
- AI infra is token-centric, but pixels, video and 3D deserve first-class status — prateekj · 2026-11-05
- 3x leveraged chip ETF SOXL hits $3.4B daily volume, topping most crypto perp markets — tomwanhh · 2026-11-05