Open-Source Project Replicates Full LLM Training Pipeline in Pure PyTorch
thisguyknowsai · x · 2026-08-15
Developer Fareed Khan released train-llm-from-scratch on GitHub, demonstrating the complete pipeline from raw data to a modern aligned reasoning model, with all algorithms hand-written in pure PyTorch, without trl, peft, or transformers. The project covers pretraining (based on Attention is All You Need), SFT, reward modeling, DPO, PPO, and GRPO (DeepSeek-R1 method), along with theory docs, hand-drawn diagrams, and code. It serves as a comprehensive educational resource for LLM training.
More from coding & agent
- MathCode Agent converts natural language math into Lean 4 proofs — tom_doerr · 2026-08-15
- Open source mcp-md-reader: Optimizes Markdown reading for Agents — JoseEstevez22 · 2026-08-15
- New Codex plugin optimizes long-task polling, reducing token waste — ssh4net · 2026-08-15
- Codex Project Memory plugin: Local persistence via MCP — ssh4net · 2026-08-15
- Design philosophies compared across four AI agent frameworks — jasonkneen · 2026-08-15
- Lazar framework emphasizes controlled self-evolution via protected Rust core — jasonkneen · 2026-08-15