Fixing Qwen3.8 27B overthinking: quantization and speed tips
Pyrolistical · reddit · 2026-08-27
Addressing reports of Qwen3.8 27B "overthinking", the author shares troubleshooting and configuration tips based on hardware specs.
Core findings:
- Speed Threshold: When generation speed exceeds 30 tokens/s (tg/s), the latency from extended thinking becomes negligible.
- Quantization Config: It is recommended to keep model quantization at least Q4 and leave the KV Cache unquantized.
Benefits:
- Reduces the chance of the model entering infinite loops.
- Maintains high output quality, ensuring the extended thinking proves its worth.
This advice applies to local deployment and performance tuning scenarios.
More from Infra
- Selling B300 inference nets $40/hr vs $6/hr rental — but the math is disputed — TheZachMueller · 2026-10-15
- ASML CFO says US Match Act restricting DUV sales to China would backfire — pstAsiatech · 2026-10-15
- ASML sold out through 2028 but stock dips 2%; TL20 index up 57% YTD — TiernanRayTech · 2026-10-15
- Bloomberg breakdown: a 1GW Texas AI data center may need nearly $3B for overruns alone — rohanpaul_ai · 2026-10-15
- 7 System Design Algorithms to Know: Merkle Trees to CAP/PACELC — goyalshaliniuk · 2026-10-15
- MiniCPM-V 4.7 runs a 35B MoE multimodal model on phones with just 3GB memory — nevrekaraishwa2 · 2026-10-15