Same model, 9x cost gap: Artificial Analysis publishes cache hit rates per provider
ArtificialAnlys · x · 2026-10-14
Artificial Analysis now publishes measured cache hit rates and per-provider Cost per Task for the endpoints it tracks.
- GLM-5.3 (max) shows a 9x spread across 21 providers (25 endpoints): $0.92/task on Makora vs $8.68 on Nebius. Both serve nearly all cacheable input from cache (98% vs 100%) and charge the same output price ($4.40/M); the gap is input pricing — Makora charges $0.10/M for cached input while Nebius gives no cached-input discount on shared serverless endpoints and bills all input at $1.40.
- Cache hit rates span 41%–100%. Eight providers hit 100% (Modal, DeepInfra, Parasail, Together AI, DigitalOcean, Nebius, Novita, Z AI first-party), 15 endpoints are ≥98%; Multiverse Computing is lowest at 41%, Bitdeer at 50%.
- Methodology: rates are measured by running multi-turn agentic coding tasks and reading cached tokens from each provider's usage response; Cost per Task applies each provider's input/output/cache prices at its measured hit rate over identical token counts.
Key takeaway: agentic workloads resend full conversation history each turn, so most input tokens are cacheable — cache pricing and hit rates dominate provider cost differences. Check cache policies before choosing a host.
Related event: Same Model, 9x Cost Gap Across API Providers, Analysis Shows(2 posts)→
More from Infra
- Ayar Labs, Wiwynn and GUC demo integrated optical compute system with live telemetry at OCP — bookwormengr · 2026-10-14
- Gavin Baker concedes some AI deals were fake, cites $1.4T commitments talking point — GavinSBaker · 2026-10-14
- Argon pushes Gemma past 120k tokens/sec on a single A100 with vLLM, a 2.9x speedup — osanseviero · 2026-10-14
- Espresso now auto-optimizes Databricks Spark jobs and configs — mattturck · 2026-10-14
- Intel Diamond Rapids motherboard shown off: 16 DDR5 slots, MRDIMM at 12800MT/s — firstadopter · 2026-10-14
- SayGM claims 70% cheaper LLM gateway with $5M ARR and crypto referral rewards — markjeffrey · 2026-10-14