One row of weights fixed a reasoning loop: 1.7B loop rate cut 92.9%, GSM8K doubles

opUserZero · reddit · 2026-11-14

A developer used direct weight editing (no training, no sampling tweaks) to fix Spark-X2.6's "but wait…" reasoning spiral by editing a single row of the output head.

The problem: the model reaches the correct answer, then refuses to commit — re-deriving conclusions and re-enumerating tools in a semantic loop invisible to standard repetition metrics, fatal in agent settings.

Diagnosis & surgery: activation inspection showed closure (</think>) was nearly absent (rank 54, P≈7×10⁻⁶) at conclusion states while restart vocabulary dominated. Since logit(t)=emb[t]·h, this is a head-readout gap. The author solved for the minimum-norm row delta raising P(close) by 14 nats at conclusion states — dequantize that one row, add the delta, requantize, splice back. 4B: 2,560 weights (6×10⁻⁷ of the model); 1.7B: 2,048. Every other byte is bit-identical.

Results (12 seeds/prompt): 4B loop rate 0.750→0.3125 (-58.4%); 1.7B 0.875→0.0625 (-92.9%). General capability is unaffected — 4B mean Δ -0.002, 1.7B +0.047 with GSM8K doubling from 0.200 to 0.433.

GGUF checkpoints are on Hugging Face with the exact delta and a CPU-only script reproducing the checkpoint bit-for-bit. Caveat: broader agent tool-looping remains an open, separate problem.

Original post →

One row of weights fixed a reasoning loop: 1.7B loop rate cut 92.9%, GSM8K doubles

More from Models