Developers Push Inkling 2 Mini to ~200 tok/s on Dual 3090s

Developers ported Thinking Machines' newly open-sourced Inkling 2 Mini to vLLM, boosting decoding speed to about 200 tok/s on dual RTX 3090s, up from 50 tok/s out of the box, while a quantized version runs at nearly 60 tok/s on a single 4090.

2026-11-21 ~ 2026-11-22 · 2 related posts