Small sites cost up to 3x per token vs data center campuses, but routing software closes the gap
antavedissian · x · 2026-11-12
antavedissian breaks down distributed inference economics: small sites cost 1.3–3x more per token than giant campuses because GPUs sit idle — regional sites $0.22, metro pods $0.36, home nodes $0.54 per million tokens vs $0.18 for a >100MW campus. Software that routes work to whichever site is free closes most of that gap (routed home fleets drop to $0.29 at 55% utilization), which is the opportunity he's watching.
Related event: Analysis: 41% of AI inference to run on sub-20MW sites by 2031(5 posts)→
More from Infra
- Dell expands AI Data Platform: agents have a data-context problem, not a model problem — DavidLinthicum · 2026-11-12
- Potion UI's local login system turns your PC into a multi-user AI server — cocktailpeanut · 2026-11-12
- Recursive's Eureka Machine ships Flint: a sparse 40B model with 2.5B active for local agentic coding — agihouse_org · 2026-11-12
- AWS Finally Built Its Own AWS Emulator — super_linear · 2026-11-12
- Megakernel is multiprogramming for GPUs: 'GPUs are entering the CPU age' — pranjalssh · 2026-11-12
- Gemini Decision 1 costs $345 per 1M tasks, 3x pricier than leader Drex Max v1.1 — ArtificialAnlys · 2026-11-12