Qwen3-8 2.4T Hits 288k tokens/s on NVIDIA GB300 NVL72
RhubarbSimilar1683 · reddit · 2026-08-17
NVIDIA's official blog announces that the Qwen3-8 2.4T model achieves over 4K tokens per second per GPU and over 350 tokens per second per user on the NVIDIA GB300 NVL72 in FP8 precision on Day 0. Further optimizations like NVFP4 are expected to enhance performance.
More from Infra
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16
- First TPUv7 agentic benchmark on vLLM claims 1.42x perf-per-dollar over GB300 NVL72 — vllm_project · 2026-10-16
- Phison CEO calls CXMT memory scare a 'ghost story', warns of materials shortage — BenBajarin · 2026-10-16
- Ben Bajarin: The Future of Compute Is Fungible as Hyperscalers Design for Workload Flexibility — BenBajarin · 2026-10-16