Analysis: 41% of AI inference to run on sub-20MW sites by 2031

Anthony Avedissian posted a series of threads plus a long-form piece on November 12 analyzing where AI inference compute will land over the next five years. The core claim: total inference will grow more than 1000x, giant data centers cannot absorb all of it, and a large share of compute will flow to sites under 20MW—a tier that conveniently escapes the current regulatory battles targeting large data centers.

Confirmed

• Regulation: Most new rules (including those debated in last week's US midterms) apply only to sites above 20MW, leaving small sites largely unconstrained.
• Forecast data: By 2031, 41% of AI inference tokens will run below the 20MW line—24% in the centralized scenario and 54% in the distributed one; roughly 1/3 of tokens still run in hyperscale campuses and 1/4 in regional sites. Even if small sites capture only 9% of a market that has grown a thousandfold, that slice equals roughly today's entire market.
• Siting logic: Which tier handles a request comes down to three questions—is the model small enough to fit at that location (on-device/local servers), is instant response needed (voice, robotics, cars), and is the data allowed to leave that building or country? Once constraints are satisfied, inference flows to the cheapest option.
• Economics: Large campuses (>100MW) cost about $0.18 per million tokens, regional sites $0.22, metro pods $0.36, and home nodes up to 3x the cost of large campuses (roughly a 1.3–3x range), driven by GPU idle time; smart scheduling can close the gap.

Why it matters

The analysis combines regulatory arbitrage, latency requirements, and data sovereignty to sketch the industry opportunity in distributed inference: small devices, buildings, and city-level sites could become a major form of compute buildout over the next five years, rather than everything concentrating in hyperscale campuses.

2026-11-12 ~ 2026-11-12 · 5 related posts

• Where a request runs: model size, latency needs, and data borders — antavedissian · 2026-11-12
• Small AI sites under 20 MW mostly skip the permitting fights battering big data centers — antavedissian · 2026-11-12
• [source] Analyst: 41% of all AI inference will run below the 20 MW line by 2031 — antavedissian · 2026-11-12
• [source] Small sites cost up to 3x per token vs data center campuses, but routing software closes the gap — antavedissian · 2026-11-12
• [source] The next 1,000x of AI inference won't fit in giant data centers: 41% runs below 20MW — antavedissian · 2026-11-12