Paper: LLMs rewrite their own weights with no gradients, 27B model gains 6.6 points
jiqizhixin · x · 2026-11-18
Paper "Language Models Can Rewrite Their Own Weights" (arXiv:2611.21885; UIUC/CMU/Princeton/Stanford) introduces self-rewriting: a 27B Qwen3.5 model reads its own weights, diagnoses failures, and edits them via pure reasoning — no gradients, optimizer, or stronger model. Its edits add 6.6 points on 490 held-out problems and keep rising over 8 sessions. It repairs a corrupted copy (7%→70% on unseen image benchmarks, 83% of intact model). The ability emerges with scale: 2B does nothing, 4B/9B break themselves. Gains persist in weights, add no inference cost, and need only one GPU for hours — suggesting improving models may become a conversation rather than a training run.
Related event: Language Models Can Rewrite Their Own Weights Without Training(3 posts)→
More from Models
- AI agents sometimes produce unreadable but correct math — burny_tech · 2026-11-18
- Code model eval: Rust hits 95-100% accuracy as 13 languages see big gains — shuding · 2026-11-18
- Asked Opus to make a video explaining consciousness and existence — "no wiser but cool" — flowersslop · 2026-11-18
- theo retests GPT-6.2 Astra: better, but still 'directionally very bad' at debugging — airesearch12 · 2026-11-18
- Claude correctly folds ubiquitin from a plain prompt, no specialized tools needed — iScienceLuvr · 2026-11-18
- GPT-6.2 Astra Tops Bug-Hunt Coding Benchmark at 62.7/105, One-Fifth Sonnet's Cost — PawelHuryn · 2026-11-18