Gemma 4 12B on a gaming laptop: 1,926 tok/s prefill but CPU-throttled 49.4 tok/s decode
draginol · x · 2026-10-19
Running Gemma 4 12B (Q4KM, 45K context) via LM Studio on an HP Omen gaming laptop with RTX 5080 Laptop GPU, Core Ultra 7 255HX and 31GB RAM: prefill hits 1,926 tok/s, first token 916ms, accuracy 80/80 — but decode is only 49.4 tok/s, with the author noting the CPU gets heavily throttled. A useful baseline ahead of his planned comparison against the Surface Laptop Ultra's RTX Spark.
Related event: RTX 5080 laptop tested running local LLMs: 27B decode drops to 9 tok/s(4 posts)→
More from Infra
- DeepInfra's B200s sat idle at $1.49/hour even with promo videos — gharik · 2026-10-19
- SK Hynix on HBF: roughly half of HBM bandwidth but 14x the capacity — Beth_Kindig · 2026-10-19
- KV Cache Compression Makes 1M Context Manageable at Under 100GB Per User — khademinori · 2026-10-19
- Elon Musk mocks what it's like trying to rent AI compute today — elonmusk · 2026-10-19
- New blog series part 1: a visual introduction to tensor core arithmetic — akbirthko · 2026-10-19
- Weekly recap: five vendors ship agent controls at five layers, desk-side AI draws most readers — shashib · 2026-10-19