Gemma 4 12b on NVIDIA RTX Spark laptop: 31.6 tok/s decode, 100% accuracy
draginol · x · 2026-10-19
Developer draginol benchmarked the Gemma 4 12b Q4KM model on a Surface Laptop Ultra with NVIDIA's RTX Spark N1X (6144-core Blackwell GPU, 20-core CPU, 64GB RAM): 1,636 tok/s prefill at 8K context, 31.6 tok/s decode, 971ms first token with a 1K prompt, and 100% accuracy (80/80 items). A concrete look at running a 12B-class model locally on next-gen Windows on-device hardware.
More from Infra
- Paying your own API bills reveals local AI's value — auto-generating setup docs — draginol · 2026-10-19
- IBM exec: US AI leadership hinges on permitted power and domestic robotics scale vs China — ingliguori · 2026-10-19
- Matthew Green: What if all frontier AI models go down in a major incident? — matthew_d_green · 2026-10-19
- HotKV semantic cache cuts Claude tokens 70%, but answer quality dips until threshold 0.88 — HotKV-Ltd · 2026-10-19
- Critics question Anthropic compute shortage amid virtual 'scientists' claims — gerardsans · 2026-10-19
- David Sacks: payback period is the North Star metric for AI infra — CoreWeave recaps in 2.5 years on 5-year deals — DavidSacks · 2026-10-19