Real agent traffic measured at 89:1 input-to-output tokens — your provider bill is prefill, not generation

MotherMouse8132 · reddit · 2026-09-10

An inference infra operator published measurements from 24 hours and 4,100 requests of real agent traffic: 201.8M input tokens vs 2.27M output — an 89:1 ratio (avg 50,239 in, 564 out per request).

Key takeaways:

Agents re-send the full conversation plus tool outputs every step, so what you're buying from providers is context reading, not generation.
Comparing providers on output price and tok/s is the wrong column for agent work; what matters is input price, cache read price, and whether prefix caching actually hits across turns.
Same 64k context: 3.95s cold prefill vs 0.78s warm — 5x faster and billed at cache rates. In an agent loop you should hit cache nearly every turn; if not, the provider isn't caching or your framework changes the prompt head between turns (a timestamp in the system prompt will do it).
KV cost per token varies wildly by architecture: their 125B MoE uses 12 KB/token vs 176 KB/token for GLM-5.3-Flash — 15x more, deciding how many long sessions a provider can hold before queueing.

The author discloses being a provider, but the data and reasoning are valuable for model selection and framework design.

Original post →

Real agent traffic measured at 89:1 input-to-output tokens — your provider bill is prefill, not generation

More from Infra