Inkling 2 Mini hits 117 tok/s on a single 4090 via vLLM, author says ditch llama.cpp

MerePotato · reddit · 2026-11-22

A Reddit user reports local inference numbers for Inkling 2 Mini: 117 tokens per second dropping to 76 at 128k context on a single 4090 with vLLM (no dspark). Their advice: if you are having issues with this model, switching from llama.cpp to vLLM makes a big difference.

Original post →

More from Infra

Infra channel →