Stanford's full CS336 LLM course is on YouTube — homework includes FlashAttention in Triton
techNmak · x · 2026-10-21
Stanford CS336: Language Modeling from Scratch, taught by Percy Liang and Tatsunori Hashimoto, has its full Spring 2026 lecture series on YouTube (18-video playlist covering GPUs/TPUs, Triton/XLA and more).
Topics covered:
- Tokenization and Transformer architectures, attention alternatives and Mixture of Experts
- GPU architecture, memory and compute; Triton kernels and FlashAttention
- Distributed training and parallelism, scaling laws and inference optimization
- Pretraining data pipelines and evaluation, SFT, RLHF and reasoning RL
The assignments are the standout: students implement their own tokenizer, Transformer and optimizer before training a small model — including writing FlashAttention in Triton.
More from Companies & People
- Jensen Huang Highlights Harvey and Trajectory Labs' Nemotron Work at GTC — saranormous · 2026-10-21
- OpenAI CPO Kevin Weil: robotic labs for large-scale data collection are science's future — kevinweil · 2026-10-21
- Free AI Search course covers Late Interaction, MUVERA scaling and Wormhole vectors — CShorten30 · 2026-10-21
- Google is underperforming mega cap peers as few outside pick its AI as daily driver — firstadopter · 2026-10-21
- Insiders credit David Baker's talent-pulling power behind new RFD4-Proteina results — AllThingsApx · 2026-10-21
- Emil Kowalski announces departure from Linear after two years — floguo · 2026-10-21