IQ2_XXS quant + 64k context makes a 27B model spin forever instead of coding
mailto_devnull · reddit · 2026-10-23
Reddit user mailtodevnull compares their daily driver — Swift-1.5-Qwen3.8-27B GGUF at Q4KL with 128k context on llama.cpp — against an IQ2XXS, 64k-context setup on Strata.
After wiring thinking levels into QFN via a thinkingLevelMap (medium→medium), a work task made the low-quant setup spin endlessly: reasoning traces literally said "I've thought long enough, time to code the fix" before another round of discovery, and even with reasoning off it spammed tool calls without writing code. The Q4KL setup started writing the fix within minutes.
Suspected causes: Strata's smaller 64k context triggering repeated compaction; medium thinking level behaving like xhigh with second-guessing; or the very low IQ2XXS quant itself. The author asks the community for thoughts.
More from Infra
- Amazon CEO Andy Jassy: $220B capex still can't meet AI demand, AWS eyes $1T run-rate — a16z · 2026-10-23
- 2-bit quantized DeepSeek 4.1 Flash tested locally: KDL degrades but still works — bakawolf123 · 2026-10-23
- Developers now deploying 1T-parameter models on their own infrastructure — dejavucoder · 2026-10-23
- Liquid AI launches RIPL inference engine generating full LLM responses in 30ms — JosephJacks_ · 2026-10-23
- Tech infra manias historically last 4-6 years — next year is AI's fifth — oran_ge · 2026-10-23
- Cambricon revenue doubles: $1.34B in first three quarters, net profit up 131% — teortaxesTex · 2026-10-23