SGLang updates Qwen3.8-27B recipes, hitting 206 tok/s on RTX 5090
ying11231 · x · 2026-08-18
SGLang released updated recipes for running Qwen3.8-27B on RTX 5090 and RTX Pro 6000. The update adds variants for non-speculative decoding, MTP, and DSpark, alongside high throughput and low latency options. Benchmarks show 206.1 tok/s decode on a single RTX 5090 using NVFP4 plus DSpark. The recipes serve as starting points, requiring users to tune flags and configs for specific use cases.
More from Infra
- Phison CEO calls CXMT memory scare a 'ghost story', warns of materials shortage — BenBajarin · 2026-10-16
- Ben Bajarin: The Future of Compute Is Fungible as Hyperscalers Design for Workload Flexibility — BenBajarin · 2026-10-16
- AirLLM runs frontier-size models layer-by-layer: 671B on 12GB VRAM — Aiden_Tech_Ai · 2026-10-16
- NVIDIA Launches cuPotential in CUDA-X to Bring AI Agents to Semiconductor Materials Simulation — tomaszbednarz · 2026-10-16
- Q4-quantized coder model on 8-core CPU builds interactive solar system demo in one shot — IntrepidMindExplorer · 2026-10-16
- TokenSpeed MLA 2 ships for Vera Rubin: up to 3.44× faster Kimi K3 decode — zhyncs42 · 2026-10-16