MTP vs MTP+Ngram on Qwen3.8 Flash: 10% Speed but 3x Token Usage
esw123 · reddit · 2026-09-03
Testing Qwen3.8 Flash Next in Unsloth Studio, the poster finds MTP alone gives 10% higher throughput but token usage jumps from 20K to 66K due to extra thinking. They ask whether Ngram should always be enabled and how to trade off speed, cost, and quality.
More from Infra
- SGLang Summit sponsors grow from 11 to 36 in one week; Lip-Bu Tan, Lilian Weng to speak — BanghuaZ · 2026-10-18
- Yacine jokes his datacenter KPI is gigawatt-hours: get paged if not burning enough power — yacineMTB · 2026-10-18
- 1.68x inference speedup drops, directly accelerating GRPO post-training — yacineMTB · 2026-10-18
- OpenRouter is only a fraction of the total addressable market, poster argues — sloppenheimer · 2026-10-18
- Developer builds change alerts for LLM prices and retirements across Bedrock, Azure and Vertex — jsandhu0780 · 2026-10-18
- 12.3M tokens in 27 minutes for $0.14: agent session with 99.6% cache hit — teortaxesTex · 2026-10-18