Running Inkling 2 Mini on 2x Titan V (24GB) at 20-30 tok/s

WindyMiller139 · reddit · 2026-11-21

A Reddit user ran Inkling 2 Mini (UD-Q4KXL GGUF) on two Nvidia Titan V cards (24GB total, power-capped at 170W) via llama.cpp: 200 tok/s prefill and 20-30 tok/s decode with 128k context. The post includes the full llama-server command with speculative decoding (dspark Q80 draft model, --spec-draft-n-max 3), jinja templates, and MoE cache settings, ready to copy for similar low-VRAM setups.

Related event: Inkling 2 Mini local tests: fast and strong at coding, but deployment remains rough(6 posts)→

Original post →

More from Infra

Infra channel →