Puro-2B Matches Qwen2.5 Performance with $6.9K Pretraining on RTX 5090
_reachsumit · x · 2026-08-28
Puro-2B introduces an open, low-cost pretraining recipe that trains a 2B model from scratch on consumer-grade RTX 5090 GPUs at a compute cost of under $6.9K, approaching Qwen2.5-1.5B performance. The efficiency is achieved through hardware selection, FP8 training, hyperball optimization, curriculum model averaging, and a specific data recipe. The authors also derive a Puro Cost Scaling Law, suggesting comparable performance can be reached for around $4.4K.
Related event: Researchers Pretrain 2B Model on a Single RTX 5090 for Under $7,000(2 posts)→
More from Infra
- Fluidstack hires Electrical Engineer for Modular R&D to build frontier AI infrastructure — MxMnr · 2026-10-16
- Jensen Huang joins BG2 podcast: does AI revenue justify capex, '27 bottlenecks — elonmusk · 2026-10-16
- Mistral's on-device infra rack revealed: DGX Spark, Mac Studio and more — gregmushen · 2026-10-16
- NVIDIA and Sweden's Wallenberg launch Sferical One, a shared AI supercomputer — nvidia · 2026-10-16
- Next infra frontier: self-evolving systems that adapt and self-verify — _shreya_s · 2026-10-16
- Xiaomi's MiMo RL dashboard reveals $3.47M spent on just 30 training steps — bycloud · 2026-10-16