OpenRouter's 10T Daily DeepSeek Tokens Are 95% Cache Hits; One GB300 Rack Could Serve It All
AccBalanced · x · 2026-10-29
demianai's analysis shows most of the trillions of tokens on OpenRouter aren't fresh GPU work — they're agents repeatedly rereading the same conversation.
Using OpenRouter's biggest DeepSeek model (V4.1 Flash) at 10 trillion daily tokens:
- 99% is input, and 95% of that input is a cache hit
- Only 114B tokens are actually generated by the model
- 460B tokens must be processed from scratch
- The remaining 8.9T tokens are just the model being shown context it already saw — agents resend the entire session every turn, and OpenRouter counts it all
thdxr puts demand in context: OpenRouter's entire DeepSeek traffic yesterday could be served by a single GB300 rack (72 GPUs, worth $6-7M). "Even if there's 10x this demand secretly, that's still <$100M of compute." The takeaway: with cache hits this high, real GPU compute is far smaller than headline token numbers suggest.
More from Infra
- Microsoft Discloses Azure Revenue in Dollars for the First Time: $33.2 Billion — TechCrunch AI · 2026-10-29
- Supabase rolls out OrioleDB public beta, a Postgres storage engine for update-heavy tables — dshukertjr · 2026-10-29
- Nadella: Microsoft Wants Every PC to Be a 'Secure Agent Box' With Unmetered Local AI — tomwarren · 2026-10-29
- YC: The Physical AI Buildout Needs New Financing for Robots, GPUs and Data Centers — Y Combinator · 2026-10-29
- Reve publishes technical report on training infra that scaled to 10K GPUs with 10x fewer chips — cantrell · 2026-10-29
- DeepSeek Flash fingerprint change hints at migration to a new GPU cluster — teortaxesTex · 2026-10-29