RTX Spark Tested: Running LLMs Locally on Consumer Hardware

Episode 1 · Dev benchmarks NVIDIA RTX Spark with Surface hardware: 27B models run locally (2026-10-18, 11 posts)

On October 18–19, developer Brad Wardell (X account @draginol) posted a series of updates sharing hands-on results from running local LLMs on Surface hardware with the NVIDIA RTX Spark N1X (6144-core Blackwell GPU, 20-core CPU, 64GB unified memory, Windows), testing his personal favorites Qwen 3.8 27B and Gemma 4 12B (Q4KM quantization), and inviting users on X to discuss and challenge his findings.

Confirmed

• Qwen 3.8 27B (dense model) measured: prefill around 671 tok/s (1K context, with some computation skipped), decode around 22 tok/s, first-token latency about 2.4 seconds; on the RTX Spark it decoded at 19.3 tok/s, reportedly with 100% accuracy.
• Gemma 4 12B (Q4KM) on the RTX Spark: prefill 1522 tok/s, decode 31.1–31.6 tok/s.
• draginol also used local Qwen 3.8 27B on a Surface Pro Ultra to quickly build a playable space shooter mini-game (wave-based asteroid shooting with scoring and health), as a practical demo.
• A tougher Extended-mode test on the Surface Laptop showed Qwen 27B decoding at 20.7 tok/s.

Conclusions and Views

• draginol's overall take: the RTX Spark is clearly stronger than gaming laptops for local AI inference, and the 27B model is genuinely usable on the platform, though the speed experience is mediocre.
• He also notes that if you mainly run 12B or even 4B small models, NPU solutions from Qualcomm, AMD, or Intel are far cheaper and more cost-effective; gaming laptops lose on both ends—struggling with large models while being uneconomical for small ones.

Why It Matters

• This is rare third-party benchmark data for NVIDIA RTX Spark-class devices (DGX Spark-level external compute), providing real-world throughput figures for 27B and 12B models on a 64GB unified-memory consumer platform—directly useful reference for anyone considering local LLM deployment.

• Early Qwen3 27B dense model benchmark: 671 tok/s prefill, 22 tok/s decode, 2.4s first token — draginol · 2026-10-18
• Dev builds a playable game on Surface Pro Ultra using local Qwen 3.8 27B — draginol · 2026-10-18
• Hands-on: Surface Laptop Ultra + NVIDIA RTX Spark tested for local AI — draginol · 2026-10-18
• Surface Laptop Ultra with NVIDIA RTX Spark tested for local AI: "hardware build quality is insane" — draginol · 2026-10-19
• Gemma 4 12b on NVIDIA RTX Spark laptop: 31.6 tok/s decode, 100% accuracy — draginol · 2026-10-19
• Qwen3.8 27B benchmarks on NVIDIA RTX Spark: 19.3 tok/s decode, 100% accuracy — draginol · 2026-10-19
• Qwen 3.8 27B tested locally on NTX Spark (Surface Laptop Ultra) — draginol · 2026-10-19
• RTX Spark on Surface Laptop runs Qwen 27B at 20.7 tok/s in Extended benchmark — draginol · 2026-10-19
• RTX Spark crushes gaming laptops at local AI, extended-mode benchmark shows — draginol · 2026-10-19
• RTX Spark runs 27B models well, but gaming laptops lose out for local AI — draginol · 2026-10-19
• For 4B-12B models, NPU solutions beat gaming laptops on cost — draginol · 2026-10-19

Episode 2 · RTX 5080 laptop tested running local LLMs: 27B decode drops to 9 tok/s (2026-10-19, 4 posts)

Tests on an RTX 5080 laptop show local 27B models suffer from insufficient VRAM, with decode speeds of only 9.1 tok/s, while the smaller Gemma 4 12B achieves roughly 1930 tok/s prefill and 49.4 tok/s decode.

• Gemma 4 12B on a gaming laptop: 1,926 tok/s prefill but CPU-throttled 49.4 tok/s decode — draginol · 2026-10-19
• Local LLM on a gaming laptop: Qwen 27B hits 441 tok/s prefill but only 9.1 tok/s decode on RTX 5080 — draginol · 2026-10-19
• Running a 27B model locally on a gaming laptop: mobile RTX 5080 falls short — draginol · 2026-10-19
• Gemma 4 12B local benchmark: 1,937 tok/s prefill on RTX 5080 laptop — draginol · 2026-10-19