Puro-2B: pretraining a 2B model on RTX 5090 for under $5090
thu-pacman · hf · 2026-08-31
Tsinghua PACMAN's Puro-2B ("Poor Lab's Qwen2-1.5B") is a cost-efficient open-source pretraining recipe:
- Pretrains a 2B-parameter model on consumer RTX 5090 GPUs for under $5090;
- Performance nears larger baselines;
- Derives cost scaling laws and studies data curricula along the way.
Highly relevant for small teams reproducing pretraining on a budget.
More from Infra
- Analyst: 80% of neoclouds may not survive, but the demand for their GPU capacity is real — BenBajarin · 2026-10-16
- Apple and Meta partner to run Muse Spark model on linked Mac Studios — pdamodaran · 2026-10-16
- Nvidia's developer base hits 11 million, up ~3.7x from 3 million in 2021 — Beth_Kindig · 2026-10-16
- Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
- Muse-Spark open-weights drop: 816B-A42B MoE model gets 359GB MLX 3-bit build — ns123abc · 2026-10-16
- Meta launches Muse Spark Open, says it runs on two M5 Ultra Mac Studios with Apple's help — The Verge AI · 2026-10-16