MiniMind: train a 64M-param LLM from scratch in 2h for ~RMB 3 on one GPU
udmrzn · x · 2026-11-18
A repost highlighting MiniMind, an open-source project that trains an ultra-small 64M-parameter LLM entirely from scratch for about RMB 3 ($0.4) in compute and 2 hours of training on a single consumer GPU.
- Ships a minimalist architecture plus an end-to-end pipeline in plain PyTorch — no third-party high-level abstractions — covering data cleaning, pretraining, SFT, LoRA, DPO, RLHF/RLAIF (PPO/GRPO/CISPO), tool use, agentic RL, and distillation
- The smallest model on the main branch is 1/2700 the size of GPT-3, making full reproduction feasible on personal GPUs
- Extended variants include vision (MiniMind-V), multimodal Omni (MiniMind-O), a diffusion LM (MiniMind-dLM), and linear attention (MiniMind-Linear)
- 66k GitHub stars, once #1 on GitHub Trending; works as both a reproducible training pipeline and a hands-on LLM tutorial
More from coding & agent
- Microsoft's Autopilot Agent Is Built on OpenClaw, Now in Preview for Frontier Customers — pswider · 2026-11-18
- SimplePost Gives ChatGPT and Claude a Publishing Button for 10 Social Networks — haltakov · 2026-11-18
- One set of Obsidian notes feeds Claude Code, Codex and Gemini alike — dSebastien · 2026-11-18
- Obsidian Starter Kit gives Claude Code, Codex and Gemini the same persistent context — dSebastien · 2026-11-18
- AI tools start from zero at every prompt — the missing piece is persistent context — dSebastien · 2026-11-18
- 27 vibe-coded Finder clones appeared in a month, most already dead — KaKi_87 · 2026-11-18