Project Orion is training a 16B model live across three continents on heterogeneous compute
markjeffrey · x · 2026-07-28
Project Orion continues its live distributed training experiment, with a focus on heterogeneous compute and longer training runs.
- The team is currently using 4090s and 5090s, and plans to add differently shaped/sized nodes such as A6000s and A100s.
- They are integrating other providers through Bittensor-native compute routed via the subnet and expanding the roadmap for scaling.
- Peak MFU so far is 40%, and they hope to converge around 30% as they tune parameters.
- The next phase will push deeper training to test DiLoCo dynamics, ResBM effects at scale, and interruptible liquid-compute experiments.
- The quoted announcement says Orion-16B is training live across three continents on permissionless, heterogeneous infrastructure owned by no single entity.
More from Infra
- Dedicated Hardware for Running AI Agents at Scale Arrives — cyrilzakka · 2026-09-23
- Ternary Bonsai 2 27B: 5.9GB weights retain ~95% of full-precision reasoning — cephaloform · 2026-09-23
- Qwen 27B runs 24hr unattended on one RTX5090, builds full Postgres-SpringBoot-React spreadsheet app — anglepoiselife · 2026-09-23
- OpenRoboto Shift launches: decentralized egocentric video data network for robot brains — markjeffrey · 2026-09-23
- Engineer describes designing digital circuits that recycle most of their energy — MikePFrank · 2026-09-23
- Cloudflare CTO Dane Knecht makes TIME's 2026 executives list as AI crawlers hit 52% of traffic — dinasaur_404 · 2026-09-23