Running Qwen 2.4T Locally: 5 GPUs Still Can't Make It Viable
klicker0 · reddit · 2026-08-14
A hardcore user tested running the massive 2.4T parameter Qwen3.8 model locally using an extreme UD-Q10 quantized version (397GB).
- Hardware: Dual RTX 5090s + three RTX 3090s + 96GB system RAM. Even with this setup, the model's requirements exceeded the total available VRAM and RAM by double.
- Performance: The generation speed was a mere 0.25 tokens/sec (taking 11 minutes and 38 seconds for 178 tokens).
- Conclusion: This speed is practically unusable, even for overnight tasks. While having open weights for massive models is great, consumer hardware still falls far short of running a 2.4T model locally.
Related event: Developers Test Run 2.4T Parameter Qwen Model on Consumer Hardware(3 posts)→
More from Infra
- First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16