UT Austin professor adds hands-on Transformers chapter to free ML e-book
GeostatsGuy · x · 2026-11-04
Michael J. Pyrcz, professor at UT Austin, added a new Transformers chapter to his free online e-book Applied Machine Learning in Python (the 8th deep learning chapter out of 56).
The chapter builds a simple Transformer from scratch and trains/diagnoses it through two full demonstrations, covering:
- Motivation and progression from LSTMs to Transformers
- Self-attention: input/positional embeddings, Query/Key/Value, attention scores and weights
- Residual connections and feed-forward networks
- Two step-by-step demos: training on synthetic data, inspecting embeddings, visualizing attention weights and residual connections
Comes with YouTube lectures and the open-source MachineLearningDemos GitHub repo. Everything is free.
More from Research
- Agents will soon edit their own weights and memory — but who studies when to use which? — sarahcat21 · 2026-11-04
- Hand-writing all weights of a 100k-param model beats training it at spotting fake legal citations — AaronBergman18 · 2026-11-04
- Gradient-Sealed Unlearning aims to stop fine-tuning from restoring harmful model capabilities — burkov · 2026-11-04
- New method forecasts GSM8K accuracy within 3 points using 5600x less compute — kenziyuliu · 2026-11-04
- Team burns 10B tokens in 2.5 days to build Omniscience, a city-scale world model — Kuprel · 2026-11-04
- Berkeley researchers use RL to improve robot data collection with Data Inoculation — berkeley_ai · 2026-11-04