A used dual-RTX-3090 local LLM rig that isn't much faster — why?
jonejy · reddit · 2026-10-14
A developer built a local LLM rig from second-hand parts: 2×RTX 3090 24GB, Ryzen 9 5950X, B550 board, 128GB DDR4 and a 1200W PSU, running Ubuntu with a recent llama.cpp build. The second GPU sits on a riser at PCIe 3.0 x4 while the first runs PCIe 4.0 x16.
Testing Qwen 35B A3B across GGUF quantizations and context sizes revealed:
- Adding the second card helps less than expected when the model already fits on one 3090;
- Longer contexts make prefill noticeably slower;
- It's unclear how much comes from KV cache, memory bandwidth or inter-GPU communication.
The poster is asking dual-3090 owners for tokens/s numbers, layer-split settings and systematic bottleneck benchmarking tips, plus which workloads are actually worth localizing.
More from Infra
- RouterPlus pools demand so small teams get bulk-rate LLM inference pricing — rohanpaul_ai · 2026-10-14
- llama.cpp ships inside Google's Android AICore powering Gemini Nano — ggerganov · 2026-10-14
- FLAT brings open-source linear attention training kernels to TPUs, up to 3x faster — SonglinYang4 · 2026-10-14
- Artist built a 300-clip immersive exhibition with MiniMax H3, benchmarks GPU render times — komarco · 2026-10-14
- Nvidia reportedly to announce $28B AI investment in Europe this week — rchardkovacs · 2026-10-14
- RISC-V crypto accelerator SoC gets full place-and-route done with ~$2,000 of LLM tokens — IgorCarron · 2026-10-14