DGX Spark Matches B300 Servers at ~$4,700 Per PFLOP, Sparking Decentralized Pretraining Idea
jon_durbin · x · 2026-08-20
Jon Durbin runs the numbers: NVIDIA B300 servers start around $350k (8 GPUs, $43,750 each) for 9 PFLOPS dense FP4, with brutal power/cooling needs and near-zero availability—about $4,861 per PFLOP.
Meanwhile the $4,700 DGX Spark plugs into any household 15-amp outlet and delivers 1 PFLOP (FP4 sparse)—roughly $4,700 per PFLOP, essentially on par with B300.
His takeaway: decentralized pretraining on DGX Spark clusters using native sparse FP4 (ternary weights fit sparse FP4 compute nicely) should be a thing—and he's about to test it.
More from Infra
- Google's Willow chip cuts quantum errors 40x, raising Q-Day concerns for Bitcoin — jamestagg · 2026-10-16
- Vibe Coding a Data Pipeline Is Easy; Running It at Billions of Rows Is Not — Zachly · 2026-10-16
- Large data centers employ only ~50-150 permanent on-site staff, Microsoft says — brezina · 2026-10-16
- Prediction: Opus 5.5-level models on laptops by 2027, backed by Qwen 27B data — cephaloform · 2026-10-16
- Oracle down 12% in a week, nearly 50% off June high as OpenAI exposure spooks market — alvelda · 2026-10-16
- US data center moratoriums top 400 — an AI bull explains why they're actually good for AI — kevinsxu · 2026-10-16