Running Qwen 3.8 27B at 75t/s on 16GB VRAM: A Practical Guide
Kernoriordan · reddit · 2026-08-31
The author shares their experience tuning Qwen 3.8 27B on an RTX 4080/5080 (16GB VRAM), achieving an average decode speed of 75t/s (peaks at 100t/s) using a specific quantization and llama.cpp configuration.
Key Setup:
- Model: Qwen3.8-27B-i1-IQ4XS-GGUF-Smaller (Hybrid quantization designed for 16GB VRAM and Multi-Token Prediction)
- Critical Args:
- -ngl 99: Offload all layers to GPU
- -c 85000: Context length
- --spec-type draft-mtp: Enable Multi-Token Prediction speculation
- --ctk q40 / --ctv q40: Quantize KV Cache
Logs demonstrate consistent performance during long text generation.
More from Infra
- TensorSharp vs llama.cpp: Qwen 3.8 Flash Next Benchmarks — fuzhongkai · 2026-09-01
- Why did increasing context size increase speed in Llama.cpp? — satnl · 2026-09-01
- mlx-signal-processing brings 10-200x faster signal ops to Apple Silicon — TheMoonMidas · 2026-09-01
- AI inference demand surges again, supply brutally outpaced by token growth — Baconbrix · 2026-09-01
- Warp founder predicts cloud-based collaborative factories for all companies within a year — charlieholtz · 2026-09-01
- JPM: 1GW of AI Infrastructure Costs $40-45B, Frontier Labs Make ~$30B per GW — zephyr_z9 · 2026-09-01