Single RTX 5090 Benchmark: 27B Model at 616 Tok/s with 262K Context
EAccelerate_42 · x · 2026-08-26
- Configuration: Running Qwen2.5-27B on a single RTX 5090 (32GB, Blackwell).
- Performance: Achieves 616 tok/s aggregate throughput at 262K context length with 4-way concurrency, using NVFP4 weights, KV Cache, and DFlash2 (K=7) speculative decoding.
- Technical Details: Provides patches for vLLM v0.27.1, Dockerfile, and build scripts. Supports tool calling and an optional CPU vision sidecar.
- Deployment: Includes a two-command deploy process and comprehensive benchmark evidence.
More from Infra
- MiniCPM-V 4.7 runs a 35B MoE multimodal model on phones with just 3GB memory — nevrekaraishwa2 · 2026-10-15
- Nvidia Is Now Worth More Than the Entire Indian Stock Market — theteknosaur · 2026-10-15
- NVIDIA joins EU's AI in Science Enablers, offering compute, training and cloud resources — nordicinst · 2026-10-15
- Open source's hardware trap: data centers crowd out consumer GPUs, local LLMs become a privilege — Dante_77A · 2026-10-15
- NVIDIA backs EU AI in Science push as 35 supercomputers to serve 3M researchers — NVIDIA Blog · 2026-10-15
- Jensen Huang: 2027 AI bottleneck is 35GW of power, revenue justifies the capex — BG2 Pod · 2026-10-15