Open-source course teaches LLMs by having you implement one in PyTorch
roberott · reddit · 2026-08-04
A hands-on open-source course to build an LLM from scratch
The author built LLM Creator Studio, a free MIT-licensed repo meant to teach how an LLM comes together end to end by actually implementing it in PyTorch rather than just reading about it.
- 18 modules / 62 exercises with NotImplementedError stubs to fill in
- Tests compare your code against PyTorch references, such as nn.MultiheadAttention and torch.optim.AdamW
- Includes full reference solutions for every exercise
- Runs on CPU, GPU, or Mac
- Ends by training a real 9M parameter GPT that generates short coherent stories
Claude Code was used for scaffolding, but the curriculum design and implementations are the author's own. The repo is open for feedback and PRs.
More from coding & agent
- A critique says OpenCode’s agent design breaks KV cache and weakens security — JFPuget · 2026-08-04
- YC-backed Buildbox launches agent analytics for real user outcomes, not just evals — ycombinator · 2026-08-04
- Gauntlet Loops show how to push Claude Code past self-grading and into long runs — mattshumer_ · 2026-08-04
- Merge adds 100+ connectors to Agent Handler for production MCP agents — shensi · 2026-08-04
- Codex keeps running after credits expire, unlike Claude Code — JFPuget · 2026-08-04
- Meme says Codex + Electron development may need 96GB of RAM — gabriel1 · 2026-08-04