wsp writes facts directly into model weights with closed-form, gradient-free solves

kastnerkyle · x · 2026-11-09

The GitHub repo qwen0.8B-wsp (18 stars) demonstrates teaching a language model facts by writing its weights directly: give it a few phrasings of a question and the answer, and wsp.py solves for the weight change in closed form, saving a normal HF model folder—no training loop, no optimizer, no gradients.

Reads activations entering mlp.downproj at layers 17-21; per-layer update dW = R Kp^T (λ·Kn Kn^T + Kp Kp^T + μ·I)^-1, with Kn protecting keys that must not move (40 general QAs + a 10x-weighted contrast set)
Spreads edits bottom-up across 5 layers; a closed loop re-solves when the edited model drifts on its own answers (boost capped at 10x, ≤6 rounds)
Only 5 of 488 tensors differ from the base model; knobs include --layer (default 87% depth) and --spread

Original post →

wsp writes facts directly into model weights with closed-form, gradient-free solves

More from coding & agent