Dev ports Inkling 2 Mini to vLLM, hitting ~200 tok/s decode on 2x RTX 3090 vs 50 on llama.cpp

peaster_ · reddit · 2026-11-22

Inkling 2 Mini ships at 50 tok/s on llama.cpp, so peaster ported it to vLLM on Ampere GPUs using the original unquantized weights with a 196K context:

Repo and full write-up with comparisons to other Ampere ports are public.

Original post →

More from coding & agent

coding & agent channel →