Running Qwen3.8B across 3 GPUs and 130GB RAM only hits 18 T/s
MageLD · reddit · 2026-10-26
A user asks how to speed up local Qwen3.8B (FN quant) inference without new hardware. Setup: 2x unlocked RTX 3090 24GB, 1x unlocked 4070 Ti Super 16GB, 18-core CPU, 130GB DDR4, 1TB NVMe.
Measured speeds: 8–9 T/s from 4070 Ti Super + NVMe alone, 2 T/s from NVMe only, and 18 T/s with all 3 GPUs plus RAM. They're looking for layer-placement and backend optimization tips.
More from Infra
- Reddit user builds $3360 Threadripper box with 256GB DDR4 to run large LLMs locally — sitytitan · 2026-10-26
- Top 1% of AI spenders direct over a third of AI budgets to raw GPU compute, Ramp data shows — rohanpaul_ai · 2026-10-26
- Used RTX 3090s Now Top $1500 on eBay as Owner Hunts a Replacement — like_a_tensor · 2026-10-26
- Ternary 1.58-bit Nemotron 30B-A3B GGUF Runs 368K Context on 12GB GPU — pmttyji · 2026-10-26
- DeepSeek V4.1-Flash hits 49 tok/s decode on dual Radeon R9700 in local test — Longjumping-Elk-7756 · 2026-10-26
- Anthropic built Scion to replace git as AI-driven merges jumped from 2,500 to 30,000/day — bibryam · 2026-10-26