7 GitHub repos to learn LLMs: from Transformer basics to training your own models
goyalshaliniuk · x · 2026-08-19
A curated list of 7 GitHub repositories for systematically learning LLMs — from Transformer fundamentals to building and training language models yourself, without dozens of scattered tutorials.
Highlighted first is Sebastian Raschka's LLMs-from-scratch (103k stars), the official code repo for Build a Large Language Model (From Scratch): step-by-step PyTorch implementation of attention, Transformers, pretraining, and fine-tuning. Others in the thread include Karpathy's nanoGPT and minGPT (300-line minimal GPT, now semi-archived in favor of nanoGPT), plus llm.c (pure C/CUDA GPT-2 pretraining, 7% faster than PyTorch Nightly, 30.8k stars), and Hugging Face's free multilingual LLM Course and the Transformers library (164k stars).
Related event: 7 GitHub Repos to Master LLMs from Scratch(2 posts)→
More from coding & agent
- a16z's Connie Chan: hardware will ship with prompts, not drivers — giffmana · 2026-08-19
- Theo finally tries out Matt Pocock's Claude skills — mattpocockuk · 2026-08-19
- Simplifying theory pays off more than quick LLM code generation long-term — math_rachel · 2026-08-19
- Abstractions simplify your Theory — until you debug beyond their contract — math_rachel · 2026-08-19
- OpenAI and Anthropic hide more data server-side, making abstraction layers brittle — math_rachel · 2026-08-19
- Why LLMs can't make your code simpler, per Naur's "Programming as Theory Building" — math_rachel · 2026-08-19