Open-source Weight-Space Repair CLI lets agents auto-fix model weights with held-out evals
Top_Speaker_7785 · reddit · 2026-11-17
Inspired by MIT's weight-space programming paper, the author open-sourced Weight-Space Repair CLI, automating everything around weight edits:
- Multi-stage surgery: an agent can commit a fix, stack up to 4 edits, roll back, and the stack is judged against the original model (e.g., restored Qwen2.5-0.5B fluency, then added a refusal steering gate when harmful answers appeared).
- Triage: checks template/sampler/context issues first, locates quantization problems down to tensors (q40 over-refusal traced to blk.16.attnv), then goes full-precision to find the exact layer/head/neuron.
- Judge the agent can't see: every fix scored on held-out validation data, seed-matched, sign-test corrected, requires ≥5-point gains plus regression and one-time confirmation sets.
- Fix types: weight patches, steering gates, sampler settings, or stacks; patches verify tensor SHA-256, export to GGUF via llama.cpp with a judgment manifest.
- Results: fixed planted blank-reply bugs (40/0), list loops (agent chose DRY itself, validation 80%→97%), over/under-refusal, Ukrainian-answered-in-Russian, damaged weights — all on Qwen 0.5B/1.5B and Llama-3.2-1B, tested on a GTX 1650.
Repo: github.com/ayushh0110/Weight-Space-Repair
More from coding & agent
- Poolside's 'everything collapses into the model' thesis shapes new agent runtime Rex — sull · 2026-11-17
- FrontierCS 2 benchmark challenges AI agents to beat published CS research blind — arena · 2026-11-17
- VPS MCP Server lets AI models manage Docker containers via Portainer API — modelcontextprotocol · 2026-11-17
- Qualcomm demos an on-device decision model routing AI requests in 8.6ms on NPU — carrycooldude · 2026-11-17
- User's open letter to OpenAI: Dot agent degraded after a week, switching to Claude Duet — Danny7Guns · 2026-11-17
- Developer outlines quest to bring Three.js rendering quality up to AAA game engines — anselm · 2026-11-17