Running DeepSeek V4 Locally on 3x MI50: Hits 15 t/s
Kamal965 · reddit · 2026-08-02
A developer successfully ran the 90.9 GB DeepSeek V4-Flash-0731 model entirely in VRAM using 3x AMD MI50 GPUs (96 GB total) with the UD-IQ2M quantization format.
Performance:
- Text Generation: Stable at 15-16 tokens/second, never dropping below 14 t/s even during a 30K token output.
- Prompt Processing: Around 105-110 tokens/second.
Quality Test:
The author tested the model with a complex prompt to generate a 3D Rubik's Cube Canvas animation. While the model made a minor factual error (confusing memory bandwidth with PCIe 4.0 bandwidth), the author was highly impressed by the overall capability and the sheer feasibility of running such a massive model locally.
Related event: DeepSeek V4 Flash Sparks a Wave of Local Deployment Tests(26 posts)→
More from Infra
- Valtz v0.1.0.11 adds FLUX.3 support, local image gen up to 4K on Mac — TgoAI · 2026-10-16
- Baidu open-sources FlashMask: up to 1.55x faster than FlashAttention-4 for long-context training — BaiduResearch · 2026-10-16
- GYRO quant of Qwen3.8-Flash-Next tested: near Q3 quality at much smaller size — PlasticRevenue4601 · 2026-10-16
- RL train-sample systems are frontier AI's hardest infra problem — and a trade secret — brianryhuang · 2026-10-16
- Local tuning of GLM-5.3-Flash on 2×GB10 lifts decode from 49 to 58 tok/s — EAccelerate_42 · 2026-10-16
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16