DeepInfer cluster serves 155T tokens/day on ~2,400 GPUs — roughly OpenRouter's weekly volume
AccBalanced · x · 2026-10-25
0xkydo breaks down DeepSeek's V4.1 Flash deployment Q&A: a single DeepInfer cluster serves 155T tokens per day on 2,400 GPUs at $38/GPU-hour (valley pricing) — matching OpenRouter's entire weekly volume. Working backward from official figures (60K fresh input tokens/sec per prefill GPU, 14K output tokens/sec per decode GPU, 1:1 ratio, 77K avg input length, 96% cache hit rate), each prefill GPU effectively serves 1.5M input tokens/sec since only 4% of input needs fresh processing. OpenRouter charts show DeepSeek V4.1 Flash leading weekly usage at 32.8T tokens.
More from Infra
- DeepSeek-V4.1-Flash EXL3-3bpw quant runs on 2x RTX PRO 6000 — No-Paper-557 · 2026-10-25
- Homelabber finishes 8x RTX 3090 build: 192GB VRAM + 256GB RAM for local LLMs — generativist · 2026-10-25
- Beyond funding, thermodynamics is the real hard limit on AI progress — AryHHAry · 2026-10-25
- KohakUnlock removes RTX 20-series FP16→FP32 lock: 2080 Ti jumps from 59.9 to 111 TFLOPS — CeFurkan · 2026-10-25
- Aggregated 18 ComfyUI GPU-unlock benchmarks show BF16 gains dwarf INT8 — Acceptable_Put9983 · 2026-10-25
- Blogger revisits Kimi K3 hype, speculates China may release some B300 GPUs — lipeng0820 · 2026-10-25