Inkling 2 Mini IQ4_NL hits 44 tok/s on an RTX 5060 Ti 16GB — llama.cpp params included
bobaburger · reddit · 2026-11-22
Reddit user bobaburger benchmarked Inkling 2 Mini (IQ4NL GGUF from ggml-org on HF) on an RTX 5060 Ti 16GB + 32GB RAM, tuning from 12 tok/s up to 44 tok/s at 128k context without vision, on latest llama.cpp.
Reproducible settings:
- Best with q80 KV (43.34 tok/s): -ctk q80 -ctv q80 -fitt 128 --spec-type ngram-mod
- Fastest with q40 KV (44.35 tok/s): -ctk q40 -ctv q40 -fitt 64
More from Infra
- Gemini 4 Neon autonomously writes GPU kernels, 5.9x faster prefill on AMD hardware — sebkrier · 2026-11-22
- SenseNova-Loop-9B Hits 1,300+ tok/s on a Laptop RTX 5090, Benchmarked vs Qwen 27B — tebjan · 2026-11-22
- Intel's 350W D270 claims 5.9x lead over RTX PRO 6000 — but the fine print reveals benchmark games — StefanoGogioso · 2026-11-22
- A tale of two Nvidias: solid earnings vs. execs calling for Jensen's return — TechCrunch AI · 2026-11-22
- Chollet: GPU spot prices keep rising as HBM sells out through 2029 — AI compute buildout isn't over — fchollet · 2026-11-22
- DeepMind bakes speculative decoding into Gemma 4: up to 3.1x server speedups, 2.2x on smartphone GPUs — dair_ai · 2026-11-22