Gemma 4 12B local benchmark: 1,937 tok/s prefill on RTX 5080 laptop
draginol · x · 2026-10-19
draginol shared local inference results for Gemma 4 12B (it, Q4KM GGUF, 45K context) on an NVIDIA RTX 5080 Laptop GPU with an Intel Core Ultra 7 255HX and 31GB RAM:
- Prefill: 1,937 tok/s at 8K context
- Decode: 49.6 tok/s
- First token: 842ms with a 1K prompt
- Accuracy: 80/80 items correct (100%)
He says he'll compile results for more models soon.
Related event: RTX 5080 laptop tested running local LLMs: 27B decode drops to 9 tok/s(4 posts)→
More from Infra
- PyTorchCon North America 2026 Opens in San Jose With Seven Tracks — PyTorch · 2026-10-20
- Tried to buy an RTX 5090 under $10k this week — couldn't find a single one — CedricMakes · 2026-10-20
- Amazon says its AI chips will support popular coding languages, challenging Nvidia — pstAsiatech · 2026-10-20
- Q-CTRL: over 99.5% of qubits sit idle in one fault-tolerant Shor's algorithm blueprint — MJBiercuk · 2026-10-20
- Token-maxxing is dead: OpenRouter's token usage plateaus — pranshuchittora · 2026-10-20
- Nscale and Anthropic reportedly in talks for another $45 billion compute deal — TechCrunch AI · 2026-10-20