IOTA SN9 tests decentralized training of 16B model on mixed GPU clusters
bittingthembits · x · 2026-08-21
The IOTA SN9 project demonstrates progress in decentralized computing by training the Orion-16B model across 226 active nodes using mixed hardware (A100s, A6000s, RTX 4090s, etc.). The experiment challenges the necessity of perfect, homogeneous GPU clusters for large-scale training. Live metrics show 97.1K tokens/sec throughput and a loss of 3.1624.
More from Infra
- NVIDIA Details Qwen3.8-2.4T Deployment on GB300, Achieving >4K Tokens/s per GPU — PyTorch · 2026-08-21
- SpaceX launch cadence could enable 12-50 GW of space compute — teortaxesTex · 2026-08-21
- Local AI Coding Hardware Tiers: $1k Gets You the Smartest Model — nickbaumann_ · 2026-08-21
- Cerebras officer Sean Lie files to sell $153M in shares as IPO retail buyers get crushed — firstadopter · 2026-08-21
- Fable launches enterprise safeguards running on your infrastructure for data control — trq212 · 2026-08-21
- Alibaba's cloud and AI revenue jumps 45%, fastest in 22 quarters, break-even seen in 3 years — pstAsiatech · 2026-08-21