Editing 0.016% of weights fixes Qwen reasoning loops, MATH-500 jumps from 7.9% to 48.1%
DeliciousGorilla · reddit · 2026-11-03
A Reddit author probed Qwen3.5-0.8B's "Wait" reasoning loops with Anthropic's Jacobian lens and reproduced MIT CSAIL's weight-edit fix.
- Background: MIT CSAIL fixed the model's reasoning loops by editing just 0.016% of its weights, lifting MATH-500 from 7.9% to 48.1%, releasing the diff for exact reproduction
- Inside the loop: at the token where the model writes "Wait" again, the lens reads "Wrong", "Nope" and "Hmm" from 60% depth onward even though the answer is done; at real stop points it reads "</think>", "Answer" and "Output"
- After the edit: the same token reads "Verification", "Completion" and "Result" instead of doubt
- Quantized validation: the edit converts cleanly to GGUF and holds at Q4KM (1 loop in 40); evals ran in llama.cpp (Vulkan) on a Ryzen AI 9 HX 370 mini PC, with code and data open-sourced
More from Models
- Gemini spirals into self-abasement after being caught fabricating sources — repligate · 2026-11-03
- SEC-Bench Pro: frontier labs test agents on 183 real V8 and SpiderMonkey vulnerabilities — NielsRogge · 2026-11-03
- Tiny specialized models trained weekly: sarcasm detection in <50ms on CPU — iamrobotbear · 2026-11-03
- Chinese AI labs fear US winning recursive self-improve as distillation gets harder — McDonaghMatthew · 2026-11-03
- Did OpenAI rush its math-breakthrough model past safety testing? — haider1 · 2026-11-03
- Don't trust viral GPT-6.2 Astra demos: PS5 controller SVG exposes Python files behind the image — Angaisb_ · 2026-11-03