693 LLM inference throughput benchmark runs across 228 models land on LocalMaxxing
TheZachMueller · x · 2026-10-27
LocalMaxxing now hosts 693 llm-inference-bench throughput results covering 228 distinct models from 35 orgs (DeepSeek, Qwen, Meta, Google, OpenAI gpt-oss, MiniMax, Mistral and more), aiming to be a unified source of LLM throughput benchmarks.
- Validated on NVIDIA H200 and RTX PRO 6000 Server nodes with 1x/2x/4x/8x GPU configs, on both SGLang and vLLM.
- "Matrix runs" measure prefill and decode speed in one go across 7 context lengths (8K–1024K) and 8 concurrency levels, with full launch commands published.
- Sample data: GLM-5.3-Flash-NVFP4 peaks at 752.8–1,330 tok/s decode on 2x H200 NVL; results include per-config TTFT and "over capacity" boundaries.
- Community contributions welcome, with lmx CLI and shareable matrix-run pages.
More from Infra
- Cadence Q3: recurring revenue up 18% as agents pull work through EDA tools — BenBajarin · 2026-10-28
- Cerebras partners with Oracle to bring its fast AI inference to OCI — vitaliychiley · 2026-10-28
- Meta's PrivacyLib runs billions of privacy checks per second across hundreds of systems — Meta_Engineers · 2026-10-28
- Open-source project claims 97% vector DB compression: 60M chunks from 201GB to 6GB — udmrzn · 2026-10-28
- Y Combinator RFS: Air-Gapped Industries Need Frontier AI Brought Inside the Walls — ycombinator · 2026-10-28
- Y Combinator RFS: The Physical AI Buildout Needs Wall Street-Style Financing for Steel and Silicon — ycombinator · 2026-10-28