Stanford CS336 explains GPU kernels: data movement, not math, is what costs you
stanfordnlp · x · 2026-10-21
In Stanford's CS336 course, Percy Liang breaks down GPU kernels: every PyTorch primitive launches its own kernel, so gelu(x) 2 reads and writes memory twice. Fusing ops into one kernel halves memory traffic — the principle is to minimize data movement, since math is cheap. Key tricks: operator fusion and tiling (how FlashAttention avoids materializing the full attention matrix). Kernels can be written in CUDA, Triton, CUTLASS or ThunderKittens, or via torch.compile.
Related event: Stanford CS336 Lecture: GPU Kernel Optimization Centers on Memory(2 posts)→
More from Infra
- PyTorch details KTIR and Dataflow Scheduler behind IBM's Spyre accelerator backend — PyTorch · 2026-10-21
- Indie dev replaces $1k OpenAI bill with a YAML pipeline on spot GPUs and vLLM — cmm324 · 2026-10-21
- DeepSeek open-sources DeepInfer, the vLLM-based engine behind V4.1-Flash — The Decoder · 2026-10-21
- Robotics startup launches Deployments to serve fine-tuned HF checkpoints to robots in real time — arnie_hacker · 2026-10-21
- ByteDance, Alibaba, Tencent order 3M+ Nvidia RTX PRO 5500 cards worth $35B, Reuters reports — SumitGup · 2026-10-21
- TorchTPU demo: unified vLLM & SGLang backend for LLM serving on TPUs — Rob_Mulla · 2026-10-21