OpenRouter served 5.2T tokens in a day—back-of-envelope math questions where all the GPUs go
teortaxesTex · x · 2026-10-29
Dev shoucccc reports OpenRouter served 5.2T tokens yesterday, while he handled 1.8T (1/3 of that volume) on just 64x B300 and 64x M890P GPUs, asking where all GPUs actually go.
teortaxesTex does the math: a trillion tokens is a small number—DeepSeek serves 155T/day on roughly 2600 GB200s, i.e. 16 GPUs per trillion daily tokens. Depending on how you count B300s and M890Ps, performance may be similar. The implication: inference at scale needs fewer GPUs than the stockpiles suggest, raising the question of what all those GPUs are actually being used for.
More from Infra
- Microsoft Cloud tops $63B with growth accelerating to 30% YoY as Nadella touts agentic stack — elonmusk · 2026-10-29
- Google's Gary Illyes on fake Googlebot crawlers, honeypots, and the AI scraping boom — gaganghotra_ · 2026-10-29
- Apple's MLX lands kernel optimizations: 2~3x batched decoding, ~2x small-prompt prefill — awnihannun · 2026-10-29
- LessWrong proposal: burn models into fixed-function ASICs to split AI training from deployment and defuse frontier risk — LessWrong 精选 · 2026-10-29
- How one Redditor squeezed Raon-1-mini from 50 to 232 tok/s on a 16GB RTX 5070 Ti — McNiiby · 2026-10-29
- Geekerwan Tests Show Snapdragon's 2nm 5GHz Push Failed to Deliver Efficiency Gains vs Apple and MediaTek — firstadopter · 2026-10-29