Cache hits don't boost inference margins—providers profit by avoiding them, dev argues

cramforce · x · 2026-11-21

Developer Malte Ubl pushes back on the claim that cache reads are what make AI subscription business models work:

The inverse of the original thesis follows: a provider can raise margins simply by not serving cache hits—and hit rates are entirely under the provider's control. The only constraint is churn: if hit rates drop too low, users leave, so providers optimize for a profit-maximizing compromise.

Original post →

More from Infra

Infra channel →