Dev Demonstrates Decentralized Training at ~$11 per Billion Tokens

Developer jondurbin shared a decentralized Pluralis-8B training run on consumer GPUs costing about $11 per billion tokens with strong MFU, and open-sourced a 1T-token pretraining dataset, while noting GPU stability concerns.

2026-09-11 ~ 2026-09-11 · 4 related posts

• Dev shares training dashboard: ~$11/b tokens cost with 'insane' MFU — jon_durbin · 2026-09-11
• 8x RTX 5090 training run hits ~$11/b tokens amid ~50% GPU failure rates — jon_durbin · 2026-09-11
• Open 1T-token pretraining corpus released at ~$11 per billion tokens trained — jon_durbin · 2026-09-11
• Pluralis-8B decentralized training run: 98 consumer-GPU nodes at ~$11 per billion tokens — jon_durbin · 2026-09-11