MiniMind: train a 64M-param LLM from scratch in 2 hours on one RTX 3090

dr_cintas · x · 2026-11-20

MiniMind is an open-source project that lets anyone train a 64M-parameter LLM (1/2700 the size of GPT-3) from scratch in about 2 hours for roughly RMB 3 of GPU rental on a single RTX 3090.

The full pipeline covers data cleaning, pretraining, SFT, LoRA, DPO, PPO/GRPO RL, tool use, agentic RL, and distillation—all in plain PyTorch with no high-level libraries hiding the details. Extended variants include MiniMind-V (vision), MiniMind-O (multimodal omni), MiniMind-dLM (diffusion), and a linear-attention version. 85k GitHub stars; ideal for developers who've only ever called an API and want to see how an LLM is actually built.

Related event: MiniMind: Train a 64M-Parameter LLM from Scratch for ~$0.40(2 posts)→

Original post →

More from Research

Research channel →