Penalizing 'wait'/'maybe' tokens lifts Qwen3.5-4B math accuracy up to +12 points
am17an · reddit · 2026-09-28
Inspired by a Meta paper on overthinking markers, a Redditor tested logit-bias penalties of -2 on 48 hedging/backtracking tokens (wait, maybe, perhaps, hmm, however, reconsider...) across quantizations of Qwen3.5-4B in llama.cpp, on 50 random MATH-500 problems.
Results:
- BF16: 74% → 84%, reasoning tokens −19.4%
- Q80: 76% → 80%, −11.0%
- Q4KM: 60% → 66%, −14.8%
- Q3KM: 52% → 66%, −17.5%
- Q2K: 12% → 24%, −11.5%
Suppressing hesitation tokens both cuts reasoning overhead and boosts accuracy, with large gains on heavier quantizations. The post includes the full copy-pasteable logit-bias command; caveats: one model, one small test.
Related event: Penalizing "wait" and "maybe" boosts Qwen accuracy(4 posts)→
More from coding & agent
- Baseten's Guide: Evaluate Open Models for Coding Agents on Your Own PRs — baseten · 2026-11-16
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16
- ffmpeg-skill gives coding agents local video editing via 42 FFmpeg tools and MCP — kajisho5 · 2026-11-16
- Agents Now Reply to Slack Messages on Their Own, Resolving a Prod Incident Ghost-to-Ghost — iamrobotbear · 2026-11-16
- Engineer's Interview Prep Method: Use Claude to Find Which Answers Undersell Your AI Work — brandon_galang · 2026-11-16
- DeepSeek open-sources its app frontend "chat-demo", already at 3.9k stars — NielsRogge · 2026-11-16