Running LLMs on mixed AMD GPUs: 11 t/s inference speed achieved
Hyungsun · reddit · 2026-08-02
A developer on Reddit shared benchmark results running a quantized version of DeepSeek-V4-Flash-0731 on a mixed AMD GPU setup (1x Radeon 7900 XTX 24GB + 3x Instinct MI60 32GB) with 128GB DDR4 RAM.
- Hardware: Dual Intel Xeon E5-2650 v4 CPUs. The heterogenous GPU combo was a result of one MI60 failing.
- Performance: Using an unoptimized llama.cpp (ROCm backend), prompt processing hit 140 t/s (dropping to 85 t/s at 64k context), with generation speeds steady at around 11.9 t/s.
Related event: DeepSeek V4 Flash Sparks a Wave of Local Deployment Tests(26 posts)→
More from Infra
- Valtz v0.1.0.11 adds FLUX.3 support, local image gen up to 4K on Mac — TgoAI · 2026-10-16
- Baidu open-sources FlashMask: up to 1.55x faster than FlashAttention-4 for long-context training — BaiduResearch · 2026-10-16
- GYRO quant of Qwen3.8-Flash-Next tested: near Q3 quality at much smaller size — PlasticRevenue4601 · 2026-10-16
- RL train-sample systems are frontier AI's hardest infra problem — and a trade secret — brianryhuang · 2026-10-16
- Local tuning of GLM-5.3-Flash on 2×GB10 lifts decode from 49 to 58 tok/s — EAccelerate_42 · 2026-10-16
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16