Running Qwen 27B on a Single 5090: 200+ t/s Inference Benchmarks
Maleficent-Ad5999 · reddit · 2026-08-31
A user successfully ran the Qwen3.8-27B model (NVFP4 quant) on a single RTX 5090 (32GB), achieving 200+ t/s decode speeds using the ninfer engine. Benchmarks show that with 180K context and MTP (5 draft tokens) enabled, decode speed reached 200.5 tok/s. The author provides specific launch commands and benchmark results, noting that while synthetic text is easy to predict (94-100% accept), real agentic coding scenarios see 51% accept rates, dropping speed to 154 tok/s.
Related event: Single RTX 5090 Runs Qwen 27B at 200+ t/s with ninfer(2 posts)→
More from Infra
- MiniCPM-V 4.5 MoE reportedly hits 13+ tokens/s on iPhone with just 3GB memory — heyshrutimishra · 2026-10-16
- Solo-built Vulkan inference engine LaForge hits ComfyUI-level SD1.5 speed: 512px in 0.8s, 402MB container — Ichidan256 · 2026-10-16
- Trading 2 RTX 5090s for 4 R9700s: does doubled VRAM pay off? — poofph · 2026-10-16
- Running PolyStrata + Cline coding agent on a $0-fancy laptop at ~90 tok/s — rpwoerk · 2026-10-16
- TensorFold 1.0.12 runs 120B vision model on 1-2 DGX Sparks, adds Mac image input — HankYeomans · 2026-10-16
- vLLM brings frontier model local deployment to NVIDIA RTX Spark PCs — vllm_project · 2026-10-16