Used R940 and two RTX 3090s run DeepSeek V4-Flash at 33 tok/s

AbbreviationsSad5582 · reddit · 2026-08-04

DeepSeek V4-Flash runs on a used R940 and two RTX 3090s at 33 tok/s

The post benchmarks the official DeepSeek V4-Flash-0731 checkpoint on commodity used hardware, arguing that capacity matters more than raw GPU speed for this sparse MoE model.

Model size: 284B total / 13B active, with an official checkpoint footprint of 156 GB.
The setup uses a Dell PowerEdge R940 with 4× Xeon Platinum 8268 CPUs, 768 GB DDR4-2933, and 2× RTX 3090 24 GB GPUs.
The inference stack is Lvllmds4-x v2.3.8, a vLLM fork with lkmoe v2.3.1 doing NUMA-aware CPU-GPU hybrid MoE execution.
Because Ampere lacks native FP4/FP8 compute, the fork routes weights through Marlin kernels; the checkpoint is quantization-aware trained and ships routed experts in MXFP4.
Reported throughput: 33 tok/s single stream, 53–68 tok/s aggregate at 4 concurrent users, and 47–63 tok/s at 8 concurrent users.
Power draw was about 1,000 W under decode, with the two 3090s only drawing 136–145 W on average; the author estimates roughly $94/month for 24/7 operation at $0.13/kWh.
The main claim is that a used enterprise server plus second-hand GPUs can run the full checkpoint for around $6K all-in.

Original post →

Used R940 and two RTX 3090s run DeepSeek V4-Flash at 33 tok/s

More from Infra