Training Frontier Models Across Compute Islands: One Post Sparks Debate

Episode 1 · Rohan Anil shows MoE training across continents over 25Gbps links (2026-11-03, 6 posts)

On November 3, Rohan Anil—former core contributor to training algorithms at Google who helped train multiple frontier models—published a long-form piece on CoreAuto, "The Hitchhiker's Guide to Training on Compute Islands," sharing hands-on practices and views on training across distributed compute islands; several community authors quickly reshared and discussed it.

Confirmed

• The team trained a sparse MoE model across two clusters of different sizes and GPU types located nearly half a planet apart, connected by a bidirectional intercontinental link of roughly 25Gbps, showing that frontier-style MoE training is feasible under such low-bandwidth conditions.
• The article's core claim is that frontier learning algorithms should be co-developed with globally distributed compute rather than relying on a single centralized datacenter.
• It also discusses the long-term vision of space training—eventually moving frontier model training onto orbital compute islands.
• Ruth Starkman of Stanford (@ruthstarkman) reshared the article with her own commentary, and @andersonbcdefg, @omarsar0, and @eliebakouch each summarized its contents.

Why it matters

• The experiment directly addresses a long-standing question in the industry: whether frontier-grade models can be trained across regions, heterogeneous hardware, and limited bandwidth. As a core contributor to Google DeepMind's training algorithms, the author's first-hand measurements carry high credibility.
• If the low-bandwidth cross-continental training route holds, geographic concentration of compute may matter less, and compute resources scattered worldwide—and eventually in space—could be pooled for frontier model training.

• Rohan Anil's CoreAuto trains MoE across two global clusters over a 25 Gbps link — _arohan_ · 2026-11-03
• Training an MoE across continents on 25 Gbps: Core Automation's compute islands guide — eliebakouch · 2026-11-03
• Ex-Google Training Lead Rohan Anil Writes on Training Across Compute Islands, Even in Space — ruthstarkman · 2026-11-03
• Frontier training algo author trains MoE across continents on 25 Gbps — andersonbcdefg · 2026-11-03
• Rohan Anil pens writeup on space-based AI training, targeting compute islands in orbit — omarsar0 · 2026-11-03
• Training a MoE Across Compute Islands: 25 Gbps Link, Two Continents, One Model — _arohan_ · 2026-11-04

Episode 2 · Training frontier models across scattered <20MW data centers sparks moat debate (2026-11-05, 3 posts)

Rohan Anil detailed training a MoE model across two continents over a 25Gbps link, prompting Arohan to argue that federated <20MW data centers can train frontier models, leaving physical space and power siting as the only remaining moat.

• Frontier training without megacampuses: MoE model trained across <20MW clusters over 25Gbps — _arohan_ · 2026-11-05
• Arohan: if training has any moat, it's space itself — _arohan_ · 2026-11-05
• Trained an MoE across two GPU clusters 25 Gbps apart — the compute islands guide — _arohan_ · 2026-11-05