Argon Optimizes Gemma to Exceed 120K Tokens/s on a Single A100

Argon's deep optimization of Gemma 4 E2B on vLLM achieved over 120,000 tokens/second on a single A100 GPU, a 2.9x speedup. The prefill stage reached about 124k tokens/s, highlighting major gains from targeted inference optimization.

2026-10-14 ~ 2026-10-14 · 2 related posts

• Argon pushes Gemma past 120k tokens/sec on a single A100 with vLLM, a 2.9x speedup — osanseviero · 2026-10-14
• Gemma optimized to break 120k tokens/sec on a single A100 in vLLM, a ~2.9x speedup — _philschmid · 2026-10-14