Mixed NVIDIA+AMD multi-GPU LLM setup benchmarks slower than a single card
LMasterGame · reddit · 2026-10-28
A Reddit user details running llama.cpp (Vulkan) on a Radeon AI Pro R9700 32GB + RTX 4500 Ada 24GB setup: dual-card layer split consistently underperformed the single R9700 (e.g. Qwen 3.8 Q6K at 30 vs 40 tok/s, Gemma 4 QAT 32 vs 50), while tensor split gave erratic results. Bandwidth isn't the bottleneck (2GiB/s observed vs 11.45GiB/s max), so they suspect the slower Ada card or PCIe latency. Full llama.cpp config included (131k context, moe-cache-mib, batch tuning). Fun aside: the 3-year-old cards are now worth more than purchase price due to the memory crisis.
More from Infra
- 7.25M Gitee Repos Scanned, 26,400 Live Secrets Found Using Modal, TruffleHog and Kingfisher — charles_irl · 2026-10-28
- DeepSeek rerun through Fireworks reproduces results; original dip was likely a fluke — teortaxesTex · 2026-10-28
- NIST Releases Official Post-Quantum Cryptography Migration Guidance as Harvest-Now-Decrypt-Later Risk Grows — entrup · 2026-10-28
- Decision Models cut inference to ~100ms, making the retriever RAG's new latency bottleneck — CShorten30 · 2026-10-28
- Infinity: an open-source Rust agent runtime that treats concurrency as a scheduling problem — ShadajL · 2026-10-28
- vLLM PR Adds Native Nemotron-3.5-Transcribe Support: Transcription+Chat, FP8/NVFP4 — vllm_project · 2026-10-28