SageMaker AI adds cross-Region inference routing to survive regional GPU shortfalls

AWS ML Blog · rss · 2026-11-20

AWS launched cross-Region inference routing for Amazon SageMaker AI via a new inference router resource. Applications invoke one path across registered endpoints (across Regions and accounts); the router prefers its own Region and shifts new requests when local destinations fail, scoring health from 5xx errors, throttles and timeouts (not request-caused errors or latency). Traffic moves within about a minute of sustained errors and returns gradually after recovery; requests that never ran the model are retried once elsewhere. Every response carries a routing reason, health states publish to CloudWatch, thresholds are managed by SageMaker, and endpoints stay unchanged. Salesforce is using it for global-scale, cross-account model serving.

Original post →

More from Infra

Infra channel →