Running 360k context DeepSeek on 4x RTX 3060 at ~100 tok/s
syscomua · reddit · 2026-08-18
A detailed technical report on running the DeepSeek V4 Flash Q4KXL model (144GB) across four RTX 3060 12GB GPUs.
By tweaking -ncmoe and tensor split parameters, the user achieved 99.4 tok/s prompt processing with a 368k context window. The post highlights that microbatch size (-ub) was the biggest performance lever and documents the specific VRAM constraints and layout strategies.
More from Infra
- First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16