Paper: LLMs rewrite their own weights with no gradients, 27B model gains 6.6 points

jiqizhixin · x · 2026-11-18

Paper "Language Models Can Rewrite Their Own Weights" (arXiv:2611.21885; UIUC/CMU/Princeton/Stanford) introduces self-rewriting: a 27B Qwen3.5 model reads its own weights, diagnoses failures, and edits them via pure reasoning — no gradients, optimizer, or stronger model. Its edits add 6.6 points on 490 held-out problems and keep rising over 8 sessions. It repairs a corrupted copy (7%→70% on unseen image benchmarks, 83% of intact model). The ability emerges with scale: 2B does nothing, 4B/9B break themselves. Gains persist in weights, add no inference cost, and need only one GPU for hours — suggesting improving models may become a conversation rather than a training run.

Related event: Language Models Can Rewrite Their Own Weights Without Training(3 posts)→

Original post →

More from Models

Models channel →