Counterintuitive: capping Qwen3.8-27B's thinking budget lifts GPQA score from 115 to 160

AIFlow_ML · x · 2026-11-18

A chart circulating from @niklaslenzai: same Qwen3.8-27B on TensorFold, GPQA Diamond (out of 198), with the only variable being a --thinking-budget cap:

Capping thinking didn't hurt accuracy — it improved it substantially, consistently across three different hardware setups (measured by the source, not re-run by the reposter). Takeaway: longer thinking isn't always better; worth trying a thinking budget on local deployments.

Original post →

More from Models

Models channel →