Hugging Face's Ultra-Scale Playbook: the best book on training LLMs at scale
mdancho84 · x · 2026-10-28
mdancho84 calls Hugging Face's "The Ultra-Scale Playbook: Training LLMs on GPU Clusters" arguably one of the best technical books on large-scale LLM training. It covers GPU memory and profiling, tiling, kernel fusion, FlashAttention, and Data/Tensor/Pipeline/Context parallelism; the PDF is freely available online. Photo shows a print edition authored by the HF team including Thomas Wolf.
More from Infra
- Harmonic's Aristotle Proves 185 Properties of a 12.2M-Gate Chip Block, Cuts Power ~26% — satnam6502 · 2026-10-29
- IT vendor lists $60M AI supercomputer with 9,600 A100 GPUs on eBay — free shipping included — giveen · 2026-10-29
- ENGY claims ~30% cheaper inference via Bittensor with no quality loss on open models — markjeffrey · 2026-10-29
- Pinokio author shares guide to saving tons of disk space with cache cleanup — cocktailpeanut · 2026-10-29
- Hugging Face's Text Embeddings Inference passes 5,000 GitHub stars, multimodal support coming — tomaarsen · 2026-10-29
- Lossless compression of HNSW proximity graph indexes cuts ANN search memory via SIMD codecs — _reachsumit · 2026-10-29