Qwen 3.8 27B at 300-500 tok/s on a 5090 laptop, tuned for parallel agents
tebjan · reddit · 2026-11-17
A career GPU programmer shares his tuning of Qwen 3.8 27B on an RTX 5090 laptop: 900 tok/s single-stream, 300-500 tok/s when optimized for parallel agent load.
- nvfp4 weights + KV cache: 4 slots with 196k context each
- Engine uses speculative decoding, internal caches, and extensive kernel tweaks
- Real workload: 4 code-review agents sweep a codebase in minutes, all on one laptop
He plans to keep optimizing, arguing parallel throughput tuning is more interesting than chasing benchmarks.
More from coding & agent
- LM Studio launches Agent Remote Control for running agents on any device — MatthewBerman · 2026-11-17
- Ripple-65B Hackathon: Self-Quantization Shrinks Model from 60GB to 30GB with Little Quality Loss — Azaliamirh · 2026-11-17
- Treating agent orchestration like a factory: the 7 jobs of an orchestration layer — blaizedsouza · 2026-11-17
- UCLA and AWS paper: AI agents find math shortcuts but thinking time often cancels the gains — rohanpaul_ai · 2026-11-17
- Pixel art + Gaussian splats game experiments: all code, no assets, but roadblocks ahead — algo_diver · 2026-11-17
- Fable 5.2 praised as first model that ignores unimportant feedback and tells you to ship — neuroecology · 2026-11-17