Penalizing 'wait'/'maybe' tokens lifts Qwen3.5-4B math accuracy up to +12 points
am17an · reddit · 2026-09-28
Inspired by a Meta paper on overthinking markers, a Redditor tested logit-bias penalties of -2 on 48 hedging/backtracking tokens (wait, maybe, perhaps, hmm, however, reconsider...) across quantizations of Qwen3.5-4B in llama.cpp, on 50 random MATH-500 problems.
Results:
- BF16: 74% → 84%, reasoning tokens −19.4%
- Q80: 76% → 80%, −11.0%
- Q4KM: 60% → 66%, −14.8%
- Q3KM: 52% → 66%, −17.5%
- Q2K: 12% → 24%, −11.5%
Suppressing hesitation tokens both cuts reasoning overhead and boosts accuracy, with large gains on heavier quantizations. The post includes the full copy-pasteable logit-bias command; caveats: one model, one small test.
More from coding & agent
- 100,000 sites scanned for agent readiness, growing 10x in a month — EdenEmarco177 · 2026-09-28
- Vibe coding favors games with addon systems—expect them everywhere — marktenenholtz · 2026-09-28
- AI agents' PR flood pushed this team from GitHub runners to on-prem CI — peterjliu · 2026-09-28
- Coding agents write defensive code with overly broad exception catches, and review agents still LGTM it — srchvrs · 2026-09-28
- Asked Claude for a small scrape, woke up to 'three weeks remaining' at 60 MB/s — generativist · 2026-09-28
- Dad and kids vibe code a Zelda-style open-world game with Opus 5.5 in one day — IanArawjo · 2026-09-28