NVIDIA Groq 3 LPX Hits 3,431 tok/s on Gemma 4 Long Context Benchmark
ricklamers · x · 2026-08-25
NVIDIA released a technical blog detailing the performance of the Groq 3 LPX accelerator on the Vera Rubin platform. Tested with the Gemma 4 31B model, the system achieved a median output speed of 3,431 tokens/second over a 100K context window, peaking at 10,996 tokens/second in high AL cases. The architecture leverages deterministic compiler scheduling, fine-grained compute-communication overlap, and pre-planned networking to minimize first-bit latency. Additionally, Groq 3 LPX supports various co-execution configurations—such as prefill-decode disaggregation and speculative external-drafter decoding—aiming to deliver best-in-class energy efficiency and interactivity for trillion-parameter models.
More from Infra
- Copper to optical: optical backplanes seen as necessary as racks move to optics — BenBajarin · 2026-10-16
- Polymarket prices 28% odds a US state enacts data center moratorium by year-end — Polymarket · 2026-10-16
- Pennsylvania residents reject $10,000-per-household offer for massive Amazon data center — Polymarket · 2026-10-16
- PyTorch now runs natively on AWS Trainium with zero code changes — PyTorch · 2026-10-16
- Chinese labs run on ~1/10 US compute; ByteDance pays 2-4x to hoover up SEA supply — robleclerc · 2026-10-16
- Zymtrace raises $12.2M and hires ML engineers to build agents that fix GPU bottlenecks — cfregly · 2026-10-16