Qwen3.8-27B hits 129.8 tok/s on a single NVIDIA GH200 with vLLM
MaziyarPanahi · x · 2026-08-21
Qwen3.8-27B (BF16) achieved 129.8 tokens/s on a single NVIDIA GH200. The setup utilizes vLLM and DFlash2, showing a 2.18x speedup over plain autoregressive inference and a 9.4% improvement over MTP-3 for single-stream requests.
Related event: Qwen3.8-27B Hits 129.8 tok/s on a Single GH200(3 posts)→
More from Infra
- White House AI chief discusses Empire AI, New York's shared compute infrastructure — mkratsios47 · 2026-10-16
- FT warns of 'Darwin moment' for AI borrowers as CoreWeave bonds sink to distress levels — firstadopter · 2026-10-16
- Phone as interface, home servers as the lab: a two-workflow local AI setup — EAccelerate_42 · 2026-10-16
- "A datacenter is a bank": CoreWeave drama revives calls for算力 supervision — arthurcolle · 2026-10-16
- DOT-MoE: converting dense models into sparse MoE via optimal transport — burkov · 2026-10-16
- Stanford CS336: A 7B model needs 14GB for weights but 84GB to train — stanfordnlp · 2026-10-16