Stanford's CS336: Free Course to Build LLMs from Scratch
bigaiguy · x · 2026-08-21
Stanford's CS336 course is highlighted as a highly underrated, free resource for learning to build language models from the ground up. Taught by highly cited researchers Percy Liang and Tatsu Hashimoto, the curriculum covers the full stack of LLM development.
Key Modules:
- Basics: Tokenization, byte-pair encoding, and vocabulary design.
- Architecture: Writing the Transformer architecture from raw PyTorch.
- Training: Distributed training and the pretraining loop.
- Scaling: Scaling laws and data curation.
- Alignment: Supervised fine-tuning, RLHF, DPO, and preference optimization.
- Inference: KV caching, speculative decoding, and serving.
- Evaluation: Red teaming and capability measurement.
All lectures, slides, and assignments are openly available on YouTube and the course website.
More from Companies & People
- UK AI Safety Institute appoints new Director amid global risk assessment efforts — S_OhEigeartaigh · 2026-08-21
- The Enduring Value of Math in an Age of AI — stevenstrogatz · 2026-08-21
- UK's AI Security Institute names Henry de Zoete as new Director — HZoete · 2026-08-21
- 15-day AI build fellowship kicks off: 52 builders picked from 300+ applications — tushaarmehtaa · 2026-08-21
- Swizec podcast: engineers who get business context will always get opportunities — js_craft_hq · 2026-08-21
- MIT Professor's 'Laws of Simplicity' remains the rulebook for Apple and Google designers — thisguyknowsai · 2026-08-21