AWS launches SageMaker JumpStart Model Leaderboard comparing 70+ models on quality, speed, cost

AWS ML Blog · rss · 2026-11-19

AWS launched the SageMaker JumpStart Model Leaderboard in SageMaker Studio, comparing 70+ open-weight models (Meta, Google, Qwen, IBM, Arcee and more) side by side. AWS runs BIG-Bench Hard, GPQA Diamond, and MMLU-Pro itself with one harness for comparable quality scores, and measures TTFT, throughput, and hourly instance cost by sending standardized workloads to real endpoints. You can switch use-case profiles and optimization goals — Balanced, Cost, Latency, or Throughput optimized (speculative decoding available) — then go from filters to a deployed model via a step-by-step walkthrough. Cost estimates cover SageMaker instance rates only.

Original post →

More from Infra

Infra channel →