Qwen3.8-27B-FP8 on GH200: 10 concurrent streaming requests, first token in 10ms
MaziyarPanahi · x · 2026-08-15
Lambda user MaziyarPanahi benchmarks Qwen3.8-27B-FP8 on a single NVIDIA GH200 with vLLM: 10 real concurrent streaming requests, each with 16K max output and 262K context. First streamed events arrived within 10ms, and all requests completed normally.
Related event: Single GH200 runs Qwen 27B FP8: 10 concurrent streams, 10ms first token(5 posts)→
More from Infra
- Three CoreWeave co-founders have sold $3.3B in shares since IPO amid credit worries — SumitGup · 2026-10-16
- Cherry pi on a Raspberry Pi: the tiny-model rumor developers are watching — sull · 2026-10-16
- CoreWeave's debt costs falling: DDTL spreads tightened from S+9.62% to S+2.25% — BenBajarin · 2026-10-16
- SoftBank lines up US IPOs for SB Energy and AI robotics spinoff Roze, valuations top $50B — nordicinst · 2026-10-16
- TRL reproduces Modal's mixed-precision async RL: plain GRPO collapses, SC+MIS holds 0.72 — QGallouedec · 2026-10-16
- Hugging Face scales Indic pretraining with 2.8T tokens and an open data pipeline — Hugging Face Blog · 2026-10-16