Testing Weight-Space Programming: LLM Weight Edits Work but Fade Fast
Experiments on four open-source models show LLM-directed weight edits can fix real bugs with neuron-level precision, but the fixes don't persist—about 20 fine-tuning steps erase them—compared against LoRA baselines.
2026-11-07 ~ 2026-11-07 · 2 related posts
- Episode 1: MIT Paper: GPT-6 Directly Edits 0.016% of Weights, Boosting Small Model Math 6x(2026-11-03, 159 posts)
- Episode 2: Editing 0.016% of weights fixes Qwen reasoning loops, lifting MATH-500 to 48.1%(2026-11-03, 2 posts)
- Episode 3: Old hands-on guide resurfaces as MIT weight-editing claims spark interest(2026-11-03, 2 posts)
- Episode 4: MIT's Weight-Space Programming Claim Meets Wave of Independent Scrutiny(2026-11-03, 13 posts)
- Episode 5: LLM-fixed driving nets still work with weights zeroed out(2026-11-04, 2 posts)
- Episode 6: MIT's Weight-Space Programming: GPT-6 Directly Edits Other Models' Weights(2026-11-05, 9 posts)
- Episode 7: Testing Weight-Space Programming: LLM Weight Edits Work but Fade Fast(2026-11-07, 2 posts)
- Weight-space programming vs LoRA on 4 bugs: precise one-neuron patches that don't persist — kevinpl07 · 2026-11-07
- LLMs can read and patch weights, but 20 fine-tuning steps erase the edits — kevinpl07 · 2026-11-07