New Release: Build Language Models from Scratch with PyTorch
burkov · x · 2026-08-24
Andriy Burkov has released a new book titled "The Hundred-Page Language Models Book." Following the concise style of his previous works, this book guides readers through mastering language models using mathematics, illustrations, and code.
Key topics include:
- Mathematical foundations of modern ML and neural networks
- Building and training three language model architectures in Python
- Coding a Transformer language model from scratch in PyTorch
- Hands-on practice with instruction finetuning and prompt engineering
Related event: Andriy Burkov releases The Hundred-Page Language Models Book(3 posts)→
More from Research
- Trained two 16M-param models to do generative CAD with real physics — debreuil · 2026-08-24
- Claude model helps discover complex structure on S^6, solving 60-year-old math problem — Singularitarian · 2026-08-24
- Study: Agents read instructions/notes 60.5% of the time, rarely touch API docs — dair_ai · 2026-08-24
- Claude Verifies 43 Lean Modules autonomously, Tackling Theoretical Physics — Tkaraletsos · 2026-08-24
- AI fakes memory: why it gets confidently wrong without forgetting — PrajwalTomar_ · 2026-08-24
- Google's AI research agents discover 66 novel biomarkers in automated biomedical study — imjustnewatai · 2026-08-24