Paper: An Open 27B Model Can Rewrite Its Own Weights Without Training
jedisct1 · x · 2026-11-21
A new arXiv paper, Language Models Can Rewrite Their Own Weights, introduces "self-rewriting": a language model acts as its own optimizer, rewriting its weights through its own reasoning with no gradients and no training.
- Unlike earlier work where a frontier model edited a much smaller one, here an open 27B model—given its own weights, graded answers, and the ability to run itself—diagnoses its failures and rewrites the weights behind them
- The self-edit makes it 6.6 points more accurate on unseen problems; it tests edits on practice problems, keeps only what works, and accuracy rises over eight consecutive sessions
- It can also repair a damaged copy of itself, restoring over 80% of original accuracy on unseen image tasks without retraining or reading the original weights
- The ability emerges with scale: smaller models in the family make no edit or break themselves; only the largest improves and self-repairs
More from Research
- Lean Formalization of the Proof Now Covers Nearly All the Number Theory — sytelus · 2026-11-21
- Mathematician predicts superhuman AI math in 2-3 years after LLM disproves unit-distance conjecture — soumitrashukla9 · 2026-11-21
- Universal Dependencies v2.19 Released: 49 New Treebanks, 33 New Languages — stanfordnlp · 2026-11-21
- CoRL 2026 best paper SuperMimic accused of repackaging 2019-era work with slick marketing — animesh_garg · 2026-11-21
- ELLIS Institute Tübingen Brings 48 Papers, 2 Orals to Three-City NeurIPS 2026 — maksym_andr · 2026-11-21
- NanoGPT Speedrun Record Falls to 5,962s, Now 49x Faster Than Baseline — Madisonkanna · 2026-11-21