Running Qwen3.8-27B with 128K context on RTX 4000 Blackwell
mmkaywhatevers · reddit · 2026-08-29
A developer successfully ran Qwen3.8-27B on a 24GB RTX PRO 4000 Blackwell. By specifying the correct CUDA 13.3 path and enabling MTP3 speculative decoding with INT8 KV cache, they achieved 128K context support. Tests show MTP3 improved decode speed from 31 tok/s to 60 tok/s (1.9x gain), with manageable VRAM usage even at high context lengths.
More from Infra
- How Microsoft optimized Windows for Nvidia's Arm-based RTX Spark AI laptops — tomwarren · 2026-10-15
- How Microsoft optimized Windows for Nvidia's Arm-based RTX Spark laptops — The Verge AI · 2026-10-15
- Global smartphone shipments plunge 22% in worst quarterly decline on record as AI demand fuels memory costs — Polymarket · 2026-10-15
- SK Hynix drops 11% again, now below 3x FY2027 earnings as AI memory selloff deepens — JOBhakdi · 2026-10-15
- ASML Sells Zero Lithography Machines into Europe as US Hosts 75%+ of AI Compute — NinaDSchick · 2026-10-15
- Nvidia quietly reclassified SPCX as 'Hyperscale', signaling deeper neocloud reliance — RihardJarc · 2026-10-15