Four 8GB 3060 Tis run Qwen3.8-27B at 120 t/s with 150k context via tensor parallel
DontWinFrensWthSalad · reddit · 2026-09-27
A Reddit user skipped buying a new GPU and instead built a 4x RTX 3060 Ti rig (8GB VRAM each, power-limited to 110W) from old mining cards, with surprisingly strong results.
Key approach
- The crucial piece is tensor parallelism: llama.cpp doesn't support it, so they used Turboderp's ExL3 first, then moved to HyperQwen + vLLM, which supports TP=4 on Ampere cards.
- ExL3 + MTP served Swift-Qwen3.8-27B at 70 t/s with 196k context.
- With vLLM running the W4A16 fast variant: 150k context at 120 t/s in bf16; KV quantization (kv8) unlocks the full 262k context but drops speed back to 70 t/s.
- Concurrency costs almost nothing: one blazing-fast agent or two concurrent agents sharing a big context window.
The takeaway: instead of spending $1000+ on a single 3090, upgrading motherboard/CPU and reusing old cards with multi-GPU tensor parallel is a viable local inference path.
More from Infra
- UK grid power shortfall could delay a major AI project by years — rvp · 2026-09-27
- FT: OpenAI projects negative free cash flow of $278B through 2030 on compute spending — Beth_Kindig · 2026-09-27
- IIT Delhi says it has built India's first indigenously designed micro-GPU — rvp · 2026-09-27
- Musk: China Will Solve Its Compute, Lithography and Chipmaking Constraints in 2-3 Years — haider1 · 2026-09-27
- Dev argues Vercel is 'unjustifiable' now that agents can safely drive Cloudflare — generativist · 2026-09-27
- ZeroHedge's GPU ROIC math uses wrong throughput, off by 8-10x — zephyr_z9 · 2026-09-27