Inkling-2-Mini quants are a trap: 17GB weights overflow 48GB VRAM at long context

EmilPi · reddit · 2026-11-21

A r/LocalLLaMA user warns against downloading Inkling-2-Mini quants for now: the 17GB IQ4NL quant overflows 48GB of VRAM even after heavy tradeoffs, runs under 20 tok/s at 131k context, and is slower than Hy4-Flash on the same hardware. Obsolete architecture choices mean the KV cache outgrows the weights at long context; the model also overthinks more than Qwen and may be benchmaxxed, with buggy llama.cpp support still unsettled between forks.

Original post →

More from Infra

Infra channel →