Two weeks of agent logs: INT4 quantization triples Qwen 27B's reasoning loops at long context

Old_Ad_6033 · reddit · 2026-10-31

The poster compared Qwen 27B INT8 (4×3090) vs AWQ INT4 (2×3090) under a heavy agent load — same sampling, Hermes harness, loop detector, two weeks of logs each:

Model calls 17.7k→11.4k; loops per 1k calls 1.9→4.7; conversations with ≥1 loop 6%→12%; per-turn success 92%→78%.
Reasoning lengths were similar, ruling out "quantized models think longer"; every loop occurred in the thinking block — repeating "Let me write it. Wait..." for 10+ lines or restating the same analysis dozens of times.
After a loop cut, asking the model to continue re-looped 14–22% of the time (vs <1% normally) — it's context, not sampling luck.
The INT4 gap grows with context: 1.4× more loops under 40k tokens, 3.3× at 80–120k.
Practical trick: with speculative decoding, loops show as a smooth speed spike (200 vs 107 tok/s) since the draft model predicts repeated text well — a cheap watchdog signal.

HF discussions report similar 20k+ token spirals on BF16, so it's not purely quantization; INT4 just amplifies it, especially past 100k tokens. Logs saved for replay when Qwen4 27B ships.

Original post →

Two weeks of agent logs: INT4 quantization triples Qwen 27B's reasoning loops at long context

More from Infra