How to locally run a Qwen 3.8 Next model at Claude Opus level for under $4,000
ironicstatistic · reddit · 2026-09-17
A Reddit user asks for deployment advice on building a dual-GPU local server to run a Q3 quant of Qwen 3.8 Next (85GB with KV cache) at 256k context, targeting Claude Opus-class intelligence with 500 tok/s prefill and 20 tok/s decode on a $4,000 budget.
Four candidate builds:
- Cheap/old: 2-3 used Tesla V100 32GB ($2,000) — affordable but aging
- AMD Strix Halo 64GB + RTX 5060 Ti 16GB ($3,500): iGPU fast RAM for experts, discrete GPU for attention via Vulkan — likely best value
- Intel: 2× Arc Pro B70 with a vLLM fork ($3,200), but probably slower
- Premium: 2× RX 7900 XTX on ROCm ($4,100), fastest and most future-proof
His thesis: with optimizations like n-gram spec decoding, mid-tier Opus-class models are becoming the realistic local-deployment target.
More from Infra
- Running DeepSeek V4.1 on a single H200 at $5.30/hr — daluoseo · 2026-11-02
- South Korea's chip-led exports accelerate, bringing $1 trillion annual target closer — WSJ — pstAsiatech · 2026-11-02
- Amazon's $215B capex exceeds $168B operating cash flow, FCF turns -$42B — thealexbanks · 2026-11-02
- BofA: B200 and H100 GPU rental prices hit new highs, up 47% and 31% YTD — TheZachMueller · 2026-11-02
- vikhyat predicts alternating compute oversupply and undersupply on the road to AGI — TheZachMueller · 2026-11-02
- PyTorch Mac downloads hit 3.29M in October, up 20.5% MoM — LysandreJik · 2026-11-02