Orion-16B Trained Across Three Continents on Decentralized Heterogeneous GPUs
Macrocosmos began real-time training of the 16B-parameter Orion-16B model on IOTA's decentralized network, spanning three continents and orchestrating 256 heterogeneous GPUs. The project demonstrates that a permissionless, non-single-entity-owned compute fabric can sustain large-model training—and even survive over 50% node churn without stopping—marking a significant engineering validation for decentralized AI infrastructure.
Confirmed
- Scale and hardware: Orion-16B is training in real time across three continents on 256 heterogeneous GPUs, including RTX 4090 and 5090 units.
- Network properties: Officials emphasize the network is permissionless, heterogeneous, and not owned by a single entity; A6000 and A100 nodes are planned to join subsequently.
- Fault tolerance: Training continued to progress even after more than 50% of nodes went offline, validating the infrastructure's robustness.
Why it matters
- Traditional large-model training relies heavily on centralized super-data centers, whereas this project successfully integrated geographically dispersed machines of varying specs and reliability. The ability to keep training uninterrupted under extreme node loss provides strong technical evidence for lowering AI compute barriers and advancing decentralized compute networks.
2026-07-28 ~ 2026-07-30 · 5 related posts
Primary sources
- Project Orion is training a 16B model live across three continents on heterogeneous compute — markjeffrey · 2026-07-28
- Macrocosmos starts a permissionless 16B model training run across three continents — markjeffrey · 2026-07-28
- [source] Iota says Orion-16B is training live on unreliable compute across three continents — markjeffrey · 2026-07-28
- [source] Decentralized Network Trains 16B Model Across 3 Continents Using RTX 4090s — bittingthembits · 2026-07-29
- [source] Project Orion says Orion-16B kept training despite losing over half its nodes — bittingthembits · 2026-07-30