Qwen3.8 Benchmarks: MTP Settings Impact Throughput, Q4 Outperforms Q8
New-Inspection7034 · reddit · 2026-08-17
Author tested Qwen3.8-27B (GGUF) on an RTX Pro 6000 Blackwell with an agent harness, revealing counterintuitive findings on performance tuning and quantization:
1. MTP Parameters Do Not Transfer
- Reusing Qwen3.6's --spec-draft-n-max 12 tanked throughput. Lowering it to 5 increased speed from 63 to 99 tok/s.
- Draft acceptance was unstable at n-max=12 (0.28–0.69) but stable at 0.82+ at n-max=5. MTP window settings do not transfer across versions, even with identical tensor layouts.
2. Q4 Quantization Beats Q8
- UD-Q4KXL achieved 126 tok/s (26% faster than Q80) while using 10GB less VRAM.
- Quality matched or exceeded Q8 in real diagnostic tasks (Q4 even found a system prompt bug Q8 missed). Author switched to Q4 as the daily driver.
3. reasoningeffort > Quantization
- In a real bug fix, xhigh produced no code in 18 mins; medium gave a correct fix in 14; low was slower and fragile.
- Adjusting reasoning effort matters more than quantization level for outcomes.
4. Model Upgrade Exposed Tooling Bugs
- The upgrade changed failure modes from "uniformly weak" to "sharp local errors," prompting an audit that found 6 bugs in the author's harness that had been misleading all previous models.
More from Infra
- Bittensor Subnet 118 Adds Ultra-Cheap Inference, Joining Major AI Providers — markjeffrey · 2026-08-17
- Meta to rely on Nvidia Blackwell, AMD Helios in 2026, accelerate custom MTIA in 2027 — Beth_Kindig · 2026-08-17
- Stripe to Acquire OpenRouter for Over $7B, 5.4x May Valuation — rohanpaul_ai · 2026-08-17
- Wici One claims to solve local VRAM limits via NVMe offloading — Torodaddy · 2026-08-17
- Qwen3.8-27B hits 206 tok/s on single RTX 5090 via SGLang — StefanoGogioso · 2026-08-17
- antirez Optimizes DwarfStar: 170 t/s Generation and 22k tokens/s Prefill on Station — antirez · 2026-08-17