DeepSeek on $75K of hardware: 2,120 tok/s decode at 16 concurrent sessions
victormustar · x · 2026-10-28
User 0xSero benchmarks DeepSeek running locally: at 16 concurrent sessions it hits 2,120 tok/s total decode, so even multiple subagent-heavy sessions still get 120+ tok/s each. Screenshot details: 4× RTX PRO 6000 ($75K hardware), 512K context, TP4 over PCIe P2P, 15,202 tok/s C4 prefill, 146ms first token, KV cache fitting 17 512K contexts, 300W per GPU.
His takeaway: for $75K a business can give an entire office private, frontier-level performance without cloud APIs.
More from Infra
- Palmer Luckey Unveils BlueChip, Billed as First American-Made Computer in Decades — whurley · 2026-10-28
- OpenRouter's entire DeepSeek traffic could run on a single GB300 rack, worth just $6-7M — teortaxesTex · 2026-10-28
- Open-source ai-phone: local LLM answers real calls on your Mac, nothing leaves the machine — Agent007_MI9 · 2026-10-28
- Agents built a custom inference engine: AMD MI350X serves GLM 5.3 FP8 37% faster than NVIDIA B200 — yisongyue · 2026-10-28
- Why RTX 5090s cost $10k: leaked factory footage shows cards converted into AI accelerators — Different_Fix_2217 · 2026-10-28
- Anthropic's S-1 admits its data centers rely on natural gas, exposing it to fuel costs — TobyWalsh · 2026-10-28