No-training weight surgery: a Claude Code plugin that fixes repetition loops by editing transformer weights directly

AchelousAce · reddit · 2026-11-13

The author hit repetition loops in a small research model and tried fixing them by editing weights directly — no training, no decoding penalty. That became Weight-Surgery, an open-source Claude Code plugin (one skill + one PyTorch file) that lets a coding agent perform this end-to-end on any trained transformer.

What the plugin does

Scans heads (induction/copy scans, ablation and scaling), attributes feed-forward writers, splits two jobs inside one head with contrast directions, applies reversible edits.
Screens edits with dose series and random controls, validates on generation, and saves a new checkpoint with a verified diff, manifest and undo file.

Results

The scans pinpointed a single true induction head (induction score 0.425, plain copy 0.012) as the loop culprit; disabling it only cut loops from 29.4% to 25.4% because it also copies names into reading answers.
Splitting the two jobs inside the copy heads instead: loops down from 30.5% to 25.2% on 960 fresh rows, greedy loops from 41.6% to 35.3%, reading untouched (+0.01), overall loss up just 0.08.
Two more wins: editing writer weights is cheap (+0.001 loss vs. +0.32 projecting the direction out of the residual stream), and a rank-1 edit on one feed-forward layer halved loops at T=0.7 from 19.4% to 9.4%.

Repo is on GitHub — a concrete path to surgical behavior fixes without retraining.

Original post →

No-training weight surgery: a Claude Code plugin that fixes repetition loops by editing transformer weights directly

More from coding & agent