NVIDIA Groq 3 LPX hits 3,431 tokens/s output at 100K context in benchmarks
scaling01 · x · 2026-09-01
An NVIDIA technical blog covers Groq 3 LPX, the interactive inference accelerator for the Vera Rubin platform. Artificial Analysis benchmarked Gemma 4 31B at 100K context and measured a median of 3,431 output tokens/s; on the SPEED-Bench coding benchmark the system reached a median of 4,767 tokens/s with a P80 of 5,520.
The accelerator relies on a deterministic execution model with compiler-scheduled chip-to-chip communication and fine-grained compute-communication overlap. Groq 3 LPX pairs with Vera Rubin NVL72 in configurations such as prefill-decode disaggregation, attention-FFN disaggregation, and external-drafter speculative decoding.
More from Infra
- RL train-sample systems are frontier AI's hardest infra problem — and a trade secret — brianryhuang · 2026-10-16
- Local tuning of GLM-5.3-Flash on 2×GB10 lifts decode from 49 to 58 tok/s — EAccelerate_42 · 2026-10-16
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16
- Optuna creator Akiba: when 2 hyperparameters cost $20M, what's left for BO/HPO — Stefania_druga · 2026-10-16
- Amazon Proposes Per-Request Validated Evidence Caching for Production RAG — _reachsumit · 2026-10-16
- LinkedIn's TransX report: sequence transduction for recsys, +6.02% CTR in A/B tests — _reachsumit · 2026-10-16