Benchmarks Confirm ThinkingCap & Swift Cut Qwen3.8-27B Reasoning Tokens ~40% With Minimal Loss
returnity · reddit · 2026-09-25
An independent Aider eval suite comparison of ThinkingCap-Qwen3.8-27B, Swift-Qwen3.8-27B, and vanilla Qwen3.8-27B validates both fine-tunes' claims of 40% reasoning token reduction with minimal performance loss.
Key results (2 runs per model, ±2-3% error bars):
- ThinkingCap scores identically to the original (27.1% first-try / 77.6% retry pass) while median completion tokens drop from 12,547 to 7,436, and wall time nearly halves (777s vs 1,481s per case)
- Swift posts the highest first-try pass rate (30.8%) with 7,301 median tokens
Nuances:
- ThinkingCap uses 8.5% more mean tokens due to a long tail of overthinking cases; Swift reduces more uniformly
- Both spend more tokens on failures than successes, more pronounced for Swift (13.2k vs 5.9k)
- By language: ThinkingCap trails Swift 8% on C++ but leads by 4% (JavaScript) and 6% (Python)
- ThinkingCap is the only model with a perfect well-formed-diff score
More from Models
- Do typos discomfort LLMs? Tokenization quirk sparks debate — menhguin · 2026-11-11
- MiniMax H3 hits 100 days open-source: 1,472 derivative models, 148M+ downloads — linoy_tsaban · 2026-11-11
- Meituan's LongCat-2.5 trends on Hugging Face as MoE multimodal model — meituan-longcat · 2026-11-11
- Fable 5.2 hands-on: great at well-defined hard problems, best since GPT 6 Astra — rachittshah · 2026-11-11
- The next AI vendor lock-in won't be the model API, it'll be the harness — Arindam_1729 · 2026-11-11
- Leaked: OpenAI's GPT-6.2 (codename betelgeuse) reportedly weighs in at ~15T tokens, drops Thursday — iruletheworldmo · 2026-11-11