Doubling to 4× RTX 5060 Ti 16GB for local MoE inference: is it worth it?
Pear_Virtual · reddit · 2026-11-16
A local LLM build discussion: with 2× RTX 5060 Ti 16GB running Qwen3.8 Flash next IQ3S on the basalt engine at 70+ tok/s decode and 900 tok/s prefill, the poster weighs buying two more cards so all experts fit in VRAM, targeting 150+ tok/s. Setup is 64GB DDR4 on X570 with PCIe bifurcation options (8x+8x or 4x4x4x4); dual 3090s at similar price are also on the table, while 128GB DDR4 is ruled out by local overpricing.
More from Infra
- Llama Windows app hits 100 tok/s running Liquid AI's LFM Omni on an RTX Spark laptop — LysandreJik · 2026-11-16
- Sarvam launches Token Factory inference stack serving open-weight models from India — itsOmSarraf_ · 2026-11-16
- Alibaba, Tencent, ByteDance all run overseas data centers, but not every Chinese hyperscaler buys Ascend — teortaxesTex · 2026-11-16
- Analyst: Anthropic IPO won't save neoclouds — their business model was broken all along — RihardJarc · 2026-11-16
- Starlink replaces 1 Mbps VSAT across North Star's 36-vessel offshore fleet — XFreeze · 2026-11-16
- Oxide's second-gen Cloud Computer ships with Micron memory and storage; Micron invests — Sethwinterroth · 2026-11-16