Benchmarking low-thinking Qwen quants: 26-33% fewer tokens, ThinkingCap runs 23% faster on a 7900 XTX
DerTomsn · reddit · 2026-09-25
The author benchmarked two "less thinking" Qwen quants (Swift and ThinkingCap) against a regular unsloth Q4KM quant on a single 7900 XTX, across 4 scenarios with 2 runs each.
Key findings:
- Token savings are real: total tokens drop from 66k (base) to 49k (ThinkingCap, -26%) and 45k (Swift, -33%).
- Different trade-offs: ThinkingCap's prefill is oddly slow (100 t/s, TTFT 6-11s) with barely-changed decode (43 t/s); Swift has the fastest prefill (614 t/s, 1s TTFT) but decode drops 33% to 32 t/s.
- Quality holds: eval scores stay in the 84-87 band, same as base.
- Runtime: ThinkingCap finishes 23% faster than base (1087s vs 1407s) — token savings outweigh the slow prefill; Swift is break-even (1400s) due to slower decode.
Caveats: only 2 runs per quant, so variance matters; the low ThinkingCap prefill speed needs more investigation. A side-by-side comparison is available on llm-bench.io.
More from Infra
- Altman says OpenAI aims to replace Nvidia in the medium to long term — zephyr_z9 · 2026-11-11
- Chinese University Bought Nvidia H200 Chips Despite US Blacklist — pstAsiatech · 2026-11-11
- Baidu's ERNIE Evolve tops public bests on NanoGPT Speedrun, brings RSI to real inference optimization — xiaohu · 2026-11-11
- Baidu says RSI auto-optimized ERNIE inference, 6.24x decode speedup on Kunxin P800 — xiaohu · 2026-11-11
- Nvidia's supply commitments jump from $119B to $279B in one quarter, locking up 2027-28 memory — tengyanAI · 2026-11-11
- Train 27B models on a 24GB GPU: ternary optimizer cuts memory to 1 byte/weight — oguzeldereli · 2026-11-11