Tsinghua's Puro-2B: Open-Source Recipe Pretrains a 2B Model on a Single RTX 5090 for Under $7K
Tsinghua's PACMAN lab released Puro-2B, a fully open-source recipe that pretrains a 2B-parameter model from scratch on a single RTX 5090 for under $7,000, drawing wide attention as an example of algorithmic efficiency challenging brute-force compute spending.
Confirmed
- Puro-2B is built on the Qwen2-1.5B architecture (self-styled "Poor Lab's" model), with a fully open training recipe reproducible on an RTX 5090.
- Official cost figures: under $5,090 to match Qwen2-1.5B, or about $6,900 to approach Qwen2.5-1.5B performance.
- The stack combines Blockwise FP8, the MuonH optimizer, curriculum model averaging, and a data recipe, training on 1.4 trillion tokens in FP8 precision.
- Motivation: high pretraining costs make large-scale controlled experiments impractical; an open low-cost recipe enables such research.
Unconfirmed
- Retold figures vary across reposts: @burnytech mentioned a $4,400 variant, and @kalyankpl described it as training Qwen3-1.5B on a $5,000 budget; these are second-hand retellings pending verification against the official report.
Why it matters
- Reposters such as @burnytech read it as clever algorithms replacing millions of dollars of server cost, opening frontier pretraining to small teams and labs.
- The fully open recipe lets the community run systematic ablations and improvements rather than merely reproducing one model.
2026-08-31 ~ 2026-09-01 · 6 related posts
- Episode 1: Researchers Pretrain 2B Model on a Single RTX 5090 for Under $7,000(2026-08-28, 2 posts)
- Episode 2: Tsinghua's Puro-2B: Open-Source Recipe Pretrains a 2B Model on a Single RTX 5090 for Under $7K(2026-08-31, 6 posts)
Primary sources
- Puro-2B: pretraining a 2B model on RTX 5090 for under $5090 — thu-pacman ·
- Train a Qwen2-Level Model on RTX 5090 for Under $5090 — zhaoran_wang ·
- Puro-2B: Replicate Qwen2-1.5B on RTX 5090 for Under $5090 — burny_tech ·
- Train a 2B model from scratch on RTX 5090 for just $4,400 — burny_tech · 2026-08-31
- [source] Puro-2B: Replicate Qwen2-1.5B on RTX 5090 for Under $5090 — burny_tech · 2026-08-31
- [source] Train a Qwen2-Level Model on RTX 5090 for Under $5090 — zhaoran_wang · 2026-08-31
- [source] Puro-2B: pretraining a 2B model on RTX 5090 for under $5090 — thu-pacman · 2026-08-31
- Train a 2B Model for $7K: Algorithmic Efficiency Challenges Hardware Brute-Force — burny_tech · 2026-09-01
- Train Qwen3-1.5B on RTX 5090 within a $5k budget — kalyan_kpl · 2026-09-01