Project Orion is training a 16B model live across three continents on heterogeneous compute
markjeffrey · x · 2026-07-28
Project Orion continues its live distributed training experiment, with a focus on heterogeneous compute and longer training runs.
- The team is currently using 4090s and 5090s, and plans to add differently shaped/sized nodes such as A6000s and A100s.
- They are integrating other providers through Bittensor-native compute routed via the subnet and expanding the roadmap for scaling.
- Peak MFU so far is 40%, and they hope to converge around 30% as they tune parameters.
- The next phase will push deeper training to test DiLoCo dynamics, ResBM effects at scale, and interruptible liquid-compute experiments.
- The quoted announcement says Orion-16B is training live across three continents on permissionless, heterogeneous infrastructure owned by no single entity.
More from Infra
- Gemma 4 is benchmarked locally on a 48GB Mac with MLX, llama.cpp and Java 25 — rseroter · 2026-07-28
- Bittensor subnet expansion is pitched as a cheaper AI infrastructure path for companies — markjeffrey · 2026-07-28
- Modular handbook maps the hidden costs of LLM inference, from KV cache to prefill/decode splits — udmrzn · 2026-07-28
- Kimi K3 reaches Merge Gateway with U.S. inference providers and ZDR terms — shensi · 2026-07-28
- Compute, not algorithms, is the real moat in frontier AI — GavinSBaker · 2026-07-28
- Renting GPUs and open-weight models cut one AI bill from $1.2M to $100K — kimmonismus · 2026-07-28