Teknium's Hermes telemetry shows 94% of tokens are cache reads, only 0.6% uncached
Teknium · x · 2026-11-22
Teknium shared token-mix telemetry from opted-in Hermes users (excluding user-added custom providers): cache reads account for 94.11% of all tokens, cache writes 4.06%, uncached input just 0.6%, and other 1.23%.
He notes the mix would shift if custom providers with poor cache quality control were included. The numbers illustrate how heavily current agent-style workloads rely on prompt caching — most input tokens are repeated context hitting the cache.
Related event: Teknium Reveals Over 90% of Hermes User Tokens Are Cache Reads(2 posts)→
More from Infra
- John Langford weighs in on FBT note: fixed-point convergence and quantized caching — JohnCLangford · 2026-11-22
- Fewer than 10 companies can safely move tenants across multi-tenant GPU clusters, argues compute-market skeptic — AccBalanced · 2026-11-22
- Tens of thousands of compute nodes power this multi-agent swarm architecture — hwchase17 · 2026-11-22
- Teknium shares Hermes telemetry: 92.4% of tokens are cache reads — Teknium · 2026-11-22
- Musk demos real-time AI video generation on half of one AI4 TRIP accelerator — jamesdouma · 2026-11-22
- Sentdex asks for ~150 free H100s right after Martin Shkreli posted wanting to buy them — Sentdex · 2026-11-22