SageMaker AI adds cross-Region inference routing to survive regional GPU shortfalls
AWS ML Blog · rss · 2026-11-20
AWS launched cross-Region inference routing for Amazon SageMaker AI via a new inference router resource. Applications invoke one path across registered endpoints (across Regions and accounts); the router prefers its own Region and shifts new requests when local destinations fail, scoring health from 5xx errors, throttles and timeouts (not request-caused errors or latency). Traffic moves within about a minute of sustained errors and returns gradually after recovery; requests that never ran the model are retried once elsewhere. Every response carries a routing reason, health states publish to CloudWatch, thresholds are managed by SageMaker, and endpoints stay unchanged. Salesforce is using it for global-scale, cross-account model serving.
More from Infra
- xAI-linked SpaceXAI buys 96 acres in Memphis for $5.5M to expand AI datacenter — chrisgrayson · 2026-11-20
- Parasail first to deploy Positron Titan at scale, claiming 4X better cost per token — JaynitMakwana · 2026-11-20
- Ollama CEO: Enterprise customers now ask where their data goes, not where models run — ollama · 2026-11-20
- Surge in expert network inquiries signals growing fear of compute industry's regulatory exposure — mattwbaker · 2026-11-20
- Lawyer runs full DeepSeek-V4.1-Flash on 12x V100 at 157-184 tok/s, beating DGX Spark stacks — TumbleweedNew6515 · 2026-11-20
- US Physical AI Runs on Private CAPEX, Frontier Models, and Energy Flexibility — ingliguori · 2026-11-20