Inkling-2-Mini quants are a trap: 17GB weights overflow 48GB VRAM at long context
EmilPi · reddit · 2026-11-21
A r/LocalLLaMA user warns against downloading Inkling-2-Mini quants for now: the 17GB IQ4NL quant overflows 48GB of VRAM even after heavy tradeoffs, runs under 20 tok/s at 131k context, and is slower than Hy4-Flash on the same hardware. Obsolete architecture choices mean the KV cache outgrows the weights at long context; the model also overthinks more than Qwen and may be benchmaxxed, with buggy llama.cpp support still unsettled between forks.
More from Infra
- AI gateways are becoming the key control layer in production AI stacks: 8 capabilities — goyalshaliniuk · 2026-11-21
- Crowds line up to try local AI demos at NVIDIA RTX Spark launch — JFPuget · 2026-11-21
- Cloudflare launches multimodal model clairic-flash, trending on Hugging Face — Cloudflare · 2026-11-21
- Hy AMA reveals Hy4 Flash runs on 96GB VRAM + 128GB RAM, plus QAT quantization details — allegedlyalienated · 2026-11-21
- Inkling-2-Mini runs on laptops with as little as 9GB RAM, per Unsloth local guide — NielsRogge · 2026-11-21
- Only when infra resources burn out will the application layer see its spring — yangyi · 2026-11-21