Argon pushes Gemma past 120k tokens/sec on a single A100 with vLLM, a 2.9x speedup
osanseviero · x · 2026-10-14
The Argon team optimized Gemma 4 E2B on vLLM to break 120,000 tokens/sec on a single A100: 124k prefill and 111k end-to-end, a 2.9x speedup with claimed zero quality degradation. They demo it by generating 64 live Python simulations simultaneously, showing what that throughput enables in interactive workloads.
Related event: Argon Optimizes Gemma to Exceed 120K Tokens/s on a Single A100(2 posts)→
More from Infra
- AMD reveals 6th Gen EPYC "Verano" CPUs to power the AI era — AccBalanced · 2026-10-14
- Alibaba optics session at OCP summit overflows, room posts 'This room is full' sign — firstadopter · 2026-10-14
- SemiAnalysis teases TPUv7 TCO data: $0.181/M tokens vs $0.222 B200 — firstadopter · 2026-10-14
- Serverless tool calling could slash agent costs 30x by running 1000 agents per sandbox — madhavsinghal_ · 2026-10-14
- Prompt caching kills easy/hard model routing: cached big model now cheaper than small one — Lance_Saul_85 · 2026-10-14
- BofA lifts global memory TAM forecast to $2T by 2030, run rate already over $1T — Beth_Kindig · 2026-10-14