The Inference Coherence Challenge in Orbital Data Centers
aaronburnett · x · 2026-07-16
This analysis focuses on a critical constraint during the inference phase of "orbital data centers": inference network coherence. The author argues this issue is rarely discussed publicly but directly impacts the efficiency and economics of space-based computing.
Key conclusions include:
- Modern Frontier LLMs mostly use MoE architectures, allowing ground operators to distribute experts across massive GPU clusters; NVIDIA benchmarks show this can boost tokens/s by up to 1.8x.
- However, a single satellite cannot coordinate like a ground server farm. GPUs must sync within roughly 5 microseconds, and the speed of light only covers a round-trip distance of about 750 meters, creating a hard constraint for maintaining coherence at the single-satellite scale.
- There are two viable paths: flying satellites in formation to reduce inter-satellite distance to about 150 meters, or confining the model within a single satellite's compute limits while accepting performance hits.
- The author estimates that an AI satellite with power specs equivalent to a rack of GB300 / NVL72 (72 GPUs) hosting a 256 expert model would require each GPU to handle significantly more experts than ground configurations.
- Under conservative assumptions, token processing efficiency on satellites is about 44% lower than on the ground. The author calls this the "containment tax," meaning it takes roughly 1.8x the satellite GPU resources to match one unit of ground-based work.
Levers to reduce this tax include:
- Software/architecture rewrites could recover up to half of the losses.
- Co-designing models to fit a satellite's 72 GPU scale, which is the cheapest and most direct lever.
- Increasing single-satellite power density, as moving to higher-power satellite nodes could further reduce base losses.
The author predicts this co-design approach will likely emerge first on models like Grok and Composer, potentially opening up to external clients in the future.
More from Infra
- Devin adds e2b sandboxes for remote agent execution — badphilosopher · 2026-07-22
- Arbitrum fee simulation shows higher gas capacity but lower L2 revenue under ArbOS61 — tomwanhh · 2026-07-22
- NVIDIA pushes OpenUSD as the common layer for simulation and physical AI — MonaJalal_ · 2026-07-22
- SkyPilot exits stealth with $20M to unify fragmented GPU compute across five clouds — skypilot_org · 2026-07-22
- Production AI budgets include retries, routing, caching and observability—not just token prices — arx-go · 2026-07-22
- NVIDIA briefs analysts on Vera CPU and doubles down on monolithic agentic design — BenBajarin · 2026-07-22