PyTorch dev ships interactive roofline explainer with NVIDIA GPU and HBM/network analysis
ezyang · x · 2026-09-20
PyTorch core dev ezyang had Astra turn the roofline chapter of 'How to Scale Your Model' into an explorable explanation: switchable between TPU and NVIDIA GPUs, with split HBM and network bandwidth analysis. Readers can tweak batch size, hidden dims, precision (BF16/FP8/INT8), compute/bandwidth multipliers and overlap, and see live roofline bounds on computation time. The original essay is from the Google DeepMind Scaling Book (Feb 2025); adaptation sections were written with OpenAI Codex.
More from Infra
- OpenAI hardware VP details first custom chip Jalapeño and its nine-month tape-out — bigdata · 2026-09-20
- focus-llama: a llama.cpp fork implementing Declarative Attention for up to 0.71x decode time — Ok-Shower7286 · 2026-09-20
- VTrain, a Vulkan-based resident trainer, fixes memory leak and offloads more work to GPU — Savantskie1 · 2026-09-20
- vLLM ships day-0 support for Qwen-Image-2.1 with cross-step prefix KV cache — Alibaba_Qwen · 2026-09-20
- Dev builds helmstudio, an MLX-first local launcher for open models on Apple Silicon — janishar · 2026-09-20
- vLLM-Omni ships KV reuse, FP8 and CUDA Graph optimizations with Qwen — vllm_project · 2026-09-20