Curing LLM overthinking in 3 weeks: 27B reasoning median cut from 32K to 4K without losing accuracy
青稞AI · wechat · 2026-09-06
A detailed engineering writeup from the Qingke community on a three-week post-training campaign against overthinking in Qwen3.8-27B. The model wasn't unable to solve problems—it just thought endlessly on them: in GPQA, correct answers had a median reasoning of 3,627 tokens while wrong ones hit the 32,768 cap, with 26 truncations of which 22 were ultimately wrong.
The three-week arc:
- Week 1: Small-dose SFT (32–130 examples) did nothing—the data didn't specify the actual failure modes.
- Week 2: Larger SFT cut thinking by 60% but accuracy dropped from 166/198 to 148/198; RLVR only recovered to 158 while making the long-tail worse.
- Week 3: Precision data curation—894 native correct traces to protect the base, 176 same-problem failure traces repaired by Grok4.6/Opus5/K3, 835 independent teacher traces audited by GPT-5.6Sol, freezing 1,905 examples. Long traces (>8K) were only 14.2% of data but carried 47.2% of reasoning tokens, preserving deep reasoning on hard problems.
Results: Week-3 SFT hit 168/198 (84.85%) with zero timeouts. All three RLVR checkpoints failed capability checks but yielded a data asset; a final 5-step SimPO run on 110 curated preference pairs (57 pathology-repair + 53 capability-protection, with 36 rejected traces kept over 32K) punished only "not finishing the task," never length itself.
Key lesson: effective behavior training needs all three data types—native correct traces, same-problem failures, and teacher solutions. A heavily post-trained model needs targeted treatment, not bulk data.
More from Models
- Claimed GPT-6 Astra + H3 Max combo builds an insanely fast realtime Blender renderer — jfischoff · 2026-09-06
- Unverified leak: OpenAI's post-Astra model to ship as 'AGI' with real-time actions — imjustnewatai · 2026-09-06
- Early Astra hands-on: code quality and data analysis feel incremental, says developer — xeophon · 2026-09-06
- Providers wage price war over serving DeepSeek-v4-flash, user burns massive tokens for a few dollars — MaziyarPanahi · 2026-09-06
- Annoyed by 'maximum length' prompts? Refreshing the page lets you keep chatting — hi-sci-collab · 2026-09-06
- GPT-6 builds a full 3D Roguelike level in Godot using just 3% of a 20X quota — op7418 · 2026-09-06