Running Qwen 27B Q4_K_M on dual RTX 3060 with llama.cpp hits ~44-50 tok/s
jacek2023 · reddit · 2026-09-26
A Redditor tested Qwen3.8-27B UD-Q4KM on a dual RTX 3060 machine using llama.cpp's llama-server with tensor parallelism, FlashAttention, 50K context, and ngram + draft-MTP speculative decoding (max 3 draft tokens). Real-world speeds: 43-50 tok/s generation and up to 198 tok/s prompt processing, making it a usable daily driver when the author's 4x3090 rig was busy with vLLM. Full command and timing logs included — evidence older 3060s still work for local LLMs.
More from Infra
- Running Qwen2.5-Coder-3B on Android: Jupyter + Pydantic Evals on-device eval pipeline — alex_b3b · 2026-11-11
- 22% of Nvidia revenue now books to Singapore, but only ~1% is local buyers — Gregory_C_Allen · 2026-11-11
- Huawei's Kirin 9050 NPU claims offline 30B MoE on smartphones, +140% perf — AIFlow_ML · 2026-11-11
- Stratechery: Intel vs AMD's diverging earnings and Intel's need for an old-new path — Stratechery · 2026-11-11
- a16z Podcast: Mike Volpi on What the 1990s Internet Boom Teaches Us About the AI Buildout — a16z Podcast · 2026-11-11
- Alibaba files to list chip-design unit T-Head in Hong Kong to fund AI push — pstAsiatech · 2026-11-11