Hugging Face's Ultra Scale Playbook: a free book on training LLMs on GPU clusters
mdancho84 · x · 2026-09-11
Hugging Face's Ultra Scale Playbook (training LLMs on GPU clusters) is being called one of the best technical books on large-scale LLM training.
Covers:
- GPU memory and profiling
- tiling, kernel fusion, FlashAttention
- Data / Tensor / Pipeline / Context Parallelism
The PDF is free to read online — a solid resource for engineers who want to understand large-scale training systems end to end.
More from Infra
- Thermo-backed CTO says AI is in its dial-up phase, touts disaggregated compute for an ultrafast 'broadband moment' — beffjezos · 2026-10-28
- DeepSeek-V4.1-Flash Q2 runs at 41 tok/s on one RTX 3090, 90% token match vs API — musicurgy · 2026-10-28
- USearch v3 RC ships after 1,000 days, its libraries now embedded in 550K-star projects — viglovikov · 2026-10-28
- Will local-model progress outrun GPU depreciation? Reddit debates the buy-now window — Sensualities · 2026-10-28
- Dev cuts local AI inference power from 100W to 25W with only a 1.9s→2.8s slowdown — AryanEmbered · 2026-10-28
- SK Hynix posts $62.8B quarterly operating profit at 80% margin as AI memory demand booms — mhdfaran · 2026-10-28