Qwen3.8-27B hits 129.8 tok/s on one GH200; vLLM wins
MaziyarPanahi · x · 2026-08-21
Benchmarks show Qwen3.8-27B BF16 achieving 129.8 tok/s on a single NVIDIA GH200. The setup included 2,048 input tokens and 1,536 forced output tokens. For single-stream requests, vLLM + DFlash2 was the clear winner, offering 2.18x speedup over plain autoregressive and 9.4% improvement over MTP-3. Compute credits provided by LambdaAPI.
Related event: Qwen3.8-27B Hits 129.8 tok/s on a Single GH200(3 posts)→
More from Infra
- Phison CEO calls CXMT memory scare a 'ghost story', warns of materials shortage — BenBajarin · 2026-10-16
- Ben Bajarin: The Future of Compute Is Fungible as Hyperscalers Design for Workload Flexibility — BenBajarin · 2026-10-16
- AirLLM runs frontier-size models layer-by-layer: 671B on 12GB VRAM — Aiden_Tech_Ai · 2026-10-16
- NVIDIA Launches cuPotential in CUDA-X to Bring AI Agents to Semiconductor Materials Simulation — tomaszbednarz · 2026-10-16
- Q4-quantized coder model on 8-core CPU builds interactive solar system demo in one shot — IntrepidMindExplorer · 2026-10-16
- TokenSpeed MLA 2 ships for Vera Rubin: up to 3.44× faster Kimi K3 decode — zhyncs42 · 2026-10-16