Dual MI50 benchmarks show 32GB VRAM for under $300
tabletuser_blogspot · reddit · 2026-10-04
- Setup: two MI50 GPUs (16GB HBM2 each, 1TB/s bandwidth, overclockable to 1200MHz), power capped at 145W, one card's miniDP activated via firmware + xrandr custom mode. Benchmarks run with Ubuntu Vulkan build of llama.cpp (b11325).
- Key results: dense 27-31B models hit 120-167 t/s prompt processing / 12-18 t/s generation; MoE models are far faster — Qwen3.6 35B-A3B Q4KM reached 983 t/s pp512 and 47 t/s tg128, Nemotron 30B-A3B hit 864 t/s pp512.
- Cost angle: 16GB MI50s average under $150, so 32GB total VRAM at this performance level costs under $300; author used RADVPERFTEST=nogttspill with -fa on -ngl 99 and notes cooling is still unsolved. A third MI50 will be added to cover the VRAM gap.
More from Infra
- Gridline launches one-API fine-tuning, distillation and batch inference for open models — Scobleizer · 2026-11-19
- Ascend output could hit ~20M chips next year; memory and packaging, not logic, are the bottleneck — teortaxesTex · 2026-11-19
- AMD Loom boosts Qwen3.5-4B prefill up to 2.6x on Radeon 890M laptops — jfowers_amd · 2026-11-19
- Dev submits PR adding periodic checkpoints to GKE sandboxes so agents don't lose progress — yoimnotkesku · 2026-11-19
- AWS Lambda Adds GPU Support via Managed Instances for Inference and Media Workloads — DavidWells · 2026-11-19
- Nvidia Allegedly Made GPUs Artificially Scarce to Keep Prices Sky-High — GaryMarcus · 2026-11-19