2x RTX 5060 Ti local LLM setup: Gen4 CPU-direct lanes triple prefill, 70-120 t/s decode
chocofoxy · reddit · 2026-11-06
The author shares lessons from building a local inference rig with two RTX 5060 Tis.
Hardware tips:
- Use a motherboard with all Gen4 lanes direct to CPU or with PCIe bifurcation — alone this can triple prefill speed;
- Add a separate low-profile GPU for display output: driving the display off a 5060 Ti burns 2-3GB VRAM, and since layer splitting is even, the other card loses the same amount, causing uneven context splits.
Model and performance:
- On Blackwell cards, NVFP4 quants give better prefill and decode;
- Running capy-flash (formerly ninfer-tp2) + OpenCode: 70-120 t/s decode and 2200 t/s prefill at 170K context; an ideal Gen4 setup should reach 3000 t/s prefill;
- P2P only adds 5% (the 5060 Ti has only 4K CUDA cores).
Harness: OpenCode's compaction avoids the "context limit reached" errors other tools hit on long tasks — his setup has run for nearly 3 days straight.
More from Infra
- Baidu open-sources ERNIE-5.5-Flash-Base: 280B MoE with 14B active params and 1M context — kimmonismus · 2026-11-06
- Data center NIMBYism is a corporate good-neighbor problem, says ex-Kraft vet — BecauseCulture · 2026-11-06
- Used AMD MI100 home-lab test: a 3D-printed shroud cools 300W at just 2,400 rpm — Much-Farmer-2752 · 2026-11-06
- Transcribing 11.3 hours of Nixon tapes in under 3 minutes with Nemotron 3.5 for ~$0.30 — vanstriendaniel · 2026-11-06
- Intel down 46% in 3 months: the bull case rests on 18A costs down 40% and $10B extra data center revenue — tengyanAI · 2026-11-06
- AI bots hit a site 5x more than humans: 1.23M requests, ChatGPT needs ~100 fetches per visitor — marclou · 2026-11-06