Inkling 2 Mini hits 117 tok/s on a single 4090 via vLLM, author says ditch llama.cpp
MerePotato · reddit · 2026-11-22
A Reddit user reports local inference numbers for Inkling 2 Mini: 117 tokens per second dropping to 76 at 128k context on a single 4090 with vLLM (no dspark). Their advice: if you are having issues with this model, switching from llama.cpp to vLLM makes a big difference.
More from Infra
- Analysts clash over Google's TPU split: bet on DeepMind/Gemini or become the Nvidia of TPUs — vitaliychiley · 2026-11-22
- Amazon nears $7.5B deal to buy AI startup Decart, nearly double its last valuation — rohanpaul_ai · 2026-11-22
- SurrealDB scans 10+ billion rows per second with a single SurrealQL query — mattturck · 2026-11-22
- Maryland's 1,300 MW coal plant can't retire until a transmission line Sierra Club opposes gets built — paulnovosad · 2026-11-22
- Agentic workloads aren't polite batch jobs: they fan out, stall, and reshape mid-run — AccBalanced · 2026-11-22
- Solo dev makes vLLM tokenization 120x faster with Elixir in under three days — AccBalanced · 2026-11-22