Where a request runs: model size, latency needs, and data borders
antavedissian · x · 2026-11-12
Follow-up in the same thread: which tier serves a request comes down to three questions — is the model small enough to fit there, does the answer need to come back instantly (voice, robots, cars), and is the data allowed to leave the building or country. After that, it goes wherever is cheapest.
Related event: Analysis: 41% of AI inference to run on sub-20MW sites by 2031(5 posts)→
More from Infra
- Polymarket prices 28% odds of a state data center moratorium; Torus raises $500M — Polymarket · 2026-11-12
- AWS-wrapper cloud startup shuts down just one month after seed round — thedealdirector · 2026-11-12
- Utah startup Torus raises $500M for shipping container-sized AI data centers deployable in 4 months — Polymarket · 2026-11-12
- Greylock reading group on AI accelerator design: more transistors no longer mean more performance — _shreya_s · 2026-11-12
- Dennard Scaling Is Dead: Automated Chip Design Could Shrink Hardware Cycles — _shreya_s · 2026-11-12
- Two smuggling scandals put Nvidia's China denials to the test — fortune · 2026-11-12