Running Qwen 3.8 Next on six V100s: MTP nearly doubles output to 43 tok/s
Odd_Caterpillar_2994 · reddit · 2026-09-20
A Redditor shares a full writeup of getting Qwen 3.8 Next running on a 6x V100 rig (TP2 PP3):
- Troubleshooting: one card dropped to PCIe Gen1 x16 (8 hours to fix); sglang-v100 kept OOMing and pxa errored, before 1cat-vllm ran stable.
- Config: speculative decoding capped at 1 due to memory; 8.78 GiB KV cache with 531K tokens.
- Benchmarks: prefill peaks at 4,679 tok/s at 16K input; enabling MTP lifts generation from an average 22.59 to 42.70 tok/s — nearly 2x.
- Thermals: 20-minute gpu-burn stress test stabilizes at 64°C with fans at only 76%.
More from Infra
- Agent Substrate Calculator Estimates Session Resume Speed Under Load — pjausovec · 2026-11-12
- Dell expands AI Data Platform: agents have a data-context problem, not a model problem — DavidLinthicum · 2026-11-12
- SGLang Megakernel Pushes Muse Spark to 1,109 tok/s, 2.1x Over Baseline — hsu_byron · 2026-11-12
- Potion UI's local login system turns your PC into a multi-user AI server — cocktailpeanut · 2026-11-12
- AWS Finally Built Its Own AWS Emulator — super_linear · 2026-11-12
- Megakernel is multiprogramming for GPUs: 'GPUs are entering the CPU age' — pranjalssh · 2026-11-12