DeepSeek v4.1 on 4x H200: 2,400 tps, $120k for 16 concurrent users
teortaxesTex · x · 2026-11-01
A developer ran DeepSeek v4.1-flash on 4x H200 (power-capped, no quantization), serving 16 concurrent users at 150+ tok/s each for roughly $120k in hardware. teortaxesTex notes the total throughput of 2,400 tps (600 tok/s per GPU) and laments the poor scaling: DeepSeek is built for datacenters, and he hopes the upcoming nano variant changes that.
More from Infra
- Interactive visualizer lets you zoom into Esperanto's ET-SoC-1 AI chip, down to FMA instructions — yaroslavvb · 2026-11-01
- Musk: Colossus 2 to host ~1.2M Blackwell GPUs by end of year — scaling01 · 2026-11-01
- Local Maximus meme: six RTX 3090s, 256GB RAM, servant to no API — gklambauer · 2026-11-01
- Tokenomos open-sourced: rewritten vLLM frontend makes small-model inference ~3x faster on RTX 5090 — bdsqlsz · 2026-11-01
- DeepSeek Details Always-On Agent Design With External Watchdog Session for Progress Monitoring — teortaxesTex · 2026-11-01
- YASAKANI inference engine claims 1.69–2.86x speed wins over SGLang, vLLM and llama.cpp on RTX 5090 — udmrzn · 2026-11-01