Local Deployment: Running an NVIDIA and AMD GPU Together for Different Models
Curious-Pen5547 · reddit · 2026-08-06
A trade business seeks to maximize its on-hand hardware by running an NVIDIA RTX 5090 and an older AMD GPU in the same PC to serve different local AI models.
Requirements:
- Primary Model: Run a custom-tuned Qwen 27B model on the 5090 for heavy tasks.
- Secondary Model: Run a lightweight 4B model (e.g., Gemma) on the 12GB AMD GPU exclusively for simple text summarization tasks running 1-5 times a day.
- Goal: Offload simple automations to the weaker model to prevent them from queuing behind larger tasks on the 5090.
The setup explores the feasibility of heterogeneous GPU scheduling using llama.cpp.
More from Infra
- Google Cloud's Filestore Migrates to Colossus, Decoupling Capacity from IOPS — rseroter · 2026-08-06
- Testing 8x DGX Spark Nodes in Open World Multi-Agent Setup — NVIDIAAI · 2026-08-06
- Open-Source Benchmarks: RTX 5090 LLM Quants and 8GB VRAM Agentic Scores — max_paperclips · 2026-08-06
- NVIDIA Discusses Building Secure Enterprise AI with Proprietary Data — nvidia · 2026-08-06
- Chorus: Open-Source Pre-trained Model Library Enables Fast CPU Inference Without GPUs — jmschreiber91 · 2026-08-06
- Running DeepSeek V4 Locally on Spark Hardware Hits ~95 tok/s — Rasmic · 2026-08-06