Qwen3.8-27B on 2x 3090 hits 218 tok/s decode with vLLM + DFlash2 spec-decode
xjx546 · reddit · 2026-08-19
A Reddit user benchmarks Qwen3.8-27B on 2× RTX 3090 (PCIe Gen4, no NVLink, patched P2P, power-capped 220/250W) using vLLM v0.26.1rc1 + AutoRound INT4 (group 128) + a DFlash2 draft model for speculative decoding:
- Code decode hits 218.3 tok/s, narrative 120.1 tok/s; TTFT 170ms
- Prefill: 1342 tok/s @ 10k context, 628 tok/s @ 90k
- Spec-decode: 7 draft tokens, 47.8% acceptance, avg accepted length 3.35
- Peak VRAM 22.3 GB/card with zero leak; context ceiling 131k (drafter uses 13.5GB)
- Measured with the Club-3090 canonical bench suite (3 warmups + 5 runs, temp 0.6/topp 0.95/topk 20); author says more performance is on the table
- Custom vLLM changes were needed to boot; all vLLM fixes were done with Kimi K3
More from Infra
- Large data centers employ only ~50-150 permanent on-site staff, Microsoft says — brezina · 2026-10-16
- Prediction: Opus 5.5-level models on laptops by 2027, backed by Qwen 27B data — cephaloform · 2026-10-16
- Oracle down 12% in a week, nearly 50% off June high as OpenAI exposure spooks market — alvelda · 2026-10-16
- US data center moratoriums top 400 — an AI bull explains why they're actually good for AI — kevinsxu · 2026-10-16
- Microsoft Research unveils Magnetar, the fastest zkVM at 10-39x speedups and post-quantum security — jedisct1 · 2026-10-16
- Bonsai 2 2B hits 120 tok/s on iPhone 18 Pro, outpaces Qwen 3.5 2B at half the memory — cephaloform · 2026-10-16