Local LLM on a gaming laptop: Qwen 27B hits 441 tok/s prefill but only 9.1 tok/s decode on RTX 5080
draginol · x · 2026-10-19
draginol continues testing local LLMs on a gaming laptop, moving up to 27B. Screenshot shows Qwen 27B Q4KM (32K context) via GGUF on an RTX 5080 Laptop GPU with Core Ultra 7 255HX and 31GB RAM: 441 tok/s prefill at 8K context, but decode of just 9.1 tok/s and a 2.91s first token at 1K prompt — the mobile 5080 isn't quite enough to fit the model, hurting performance. His earlier 12B run was also throttled by the CPU. Takeaway: 27B is marginal on mobile hardware; local deployment still wants more VRAM.
Related event: RTX 5080 laptop tested running local LLMs: 27B decode drops to 9 tok/s(4 posts)→
More from Infra
- Hugging Face kicks off Open Source AI Week with ExecuTorch edge-AI hackathon — Thom_Wolf · 2026-10-19
- Local models are now good enough for first-round code review, dev finds while paying frontier API prices — draginol · 2026-10-19
- Dev replaces hundreds/month in Modal sandboxes with an idle Mac Mini at home — andersonbcdefg · 2026-10-19
- TensorFold 1.1.0 hits 3,962 tok/s prefill at 205k ctx on M5 Ultra — GabGarrett · 2026-10-19
- For 4B-12B models, NPU solutions beat gaming laptops on cost — draginol · 2026-10-19
- RTX Spark runs 27B models well, but gaming laptops lose out for local AI — draginol · 2026-10-19