Running Qwen3.6-35B on a 2018 GTX 1050 Ti Laptop: Self-Built CUDA 12.9 Cuts 9-Minute Prompts to 76s

Uncovered-Myth · reddit · 2026-11-08

A Reddit user got Qwen3.6-35B-A3B (Unsloth UD-Q4KXL GGUF) running as a usable Hermes agent backend on a 2018 Acer Nitro 5 (i5-8300H, GTX 1050 Ti 4GB, 32GB RAM), with a detailed reproducible writeup:

Initial pain: Vulkan gave only 22-32 tok/s prompt processing; a 10k-token prompt took 5-7 minutes. Models under 8B couldn't handle tool calls reliably.
Key fix: CUDA 13 dropped Pascal cards, so they built CUDA 12.9 with clang (nvcc couldn't parse newer GCC headers). Prompt processing jumped from 30 to 200 tok/s in benchmarks, 167 on real prompts.
Prompt caching: the model's hybrid layers support it, so each turn processes only 46 new tokens; SSD cache save/restore cut warm restart from 85s to 0.15s.
Other tweaks: micro-batch 1024, pinned memory, q50 KV cache freeing enough VRAM for 64k context, and a 512 reasoning budget to stop thinking-token loops.
Result: a cold 12.7k-token agent prompt dropped from 8-9 minutes to 76 seconds; follow-ups take 5-8 seconds. Generation is still 12 tok/s. MTP speculative decoding was tried and dropped.

A valuable pitfall log for running MoE models on old hardware for agent use.

Original post →

Running Qwen3.6-35B on a 2018 GTX 1050 Ti Laptop: Self-Built CUDA 12.9 Cuts 9-Minute Prompts to 76s

More from coding & agent