kneebench: open-source LLM benchmark that tells you what the numbers mean and which vLLM flag to tweak
s1lv3rj1nx · reddit · 2026-10-22
Developer s1lv3rj1nx released kneebench (alpha), an open-source benchmarking and tuning tool built on NVIDIA's AIPerf, aiming to solve the "running benchmarks is easy, interpreting them is hard" problem.
- Tells you how many concurrent users your server can handle at your latency targets, what breaks first, and which vLLM flag to change next
- Sample report: unsloth/Meta-Llama-3.1-8B-Instruct on one H100 NVL 94GB with vLLM 0.29.0, covering single-request baseline, concurrency sweep (1-256 users), request-rate/size sweeps, long-prompt interference, and prompt reuse — 47 test points
- Reports include actionable sections like "What needs attention" and "Try these next"
Repo: github.com/S1LV3RJ1NX/kneebench, feedback welcome.
More from Infra
- Sundar Pichai: Google Cloud could be selling quantum deployments within 5 years — rohanpaul_ai · 2026-10-22
- Sundar Pichai: Google Cloud could be selling quantum deployments at scale within 5 years — rohanpaul_ai · 2026-10-22
- TSMC surveys suppliers on rare-earth-free motors for factory automation — pstAsiatech · 2026-10-22
- Zoho cries poor over AI costs, then applies to build $200M semiconductor fab — kingslyj · 2026-10-22
- NVIDIA recaps GTC Berlin dev community: open models to physical AI — NVIDIAAI · 2026-10-22
- GPU payback math: B300 rigs recoup costs in 10 months, 1GW build could earn ~$38B a year — tctjr · 2026-10-22