Stanford-led paper: 27B open model rewrites its own weights, gains 6.6 points without training
rohanpaul_ai · x · 2026-11-21
A paper from UIUC, CMU, Stanford, Princeton and others introduces "self-rewriting": Qwen3.5-27B, given its own weights, graded practice answers and a Python session with gradients blocked, diagnoses its failures via reasoning and edits its own weights.
- Method: the model compares hidden states on answers that finished vs. answers that ran out of tokens, then weakens the direction separating them.
- Results: accuracy rises from 57.2% to 63.8% (+6.6 points) on 490 pre-fixed unseen problems; unlike test-time scaling, the gain persists in the weights at no per-answer cost.
- Repair: trained only on text math, the repair lifts a damaged copy from 7% to 70% across six unseen image benchmarks (83% of the intact model).
- Scale: the 2B model makes no edit; 4B and 9B models break themselves; only 27B succeeds.
- Practical tip: let large open models propose weight edits for their failures, keeping only edits that pass held-out tests; needs just one GPU and hours.
More from Research
- "Wright Brothers first flight for math": KOLs hail a new era after math breakthrough — jamesdouma · 2026-11-21
- Language Models Can Rewrite Their Own Weights: 27B Model Self-Improves Without Gradients — Jiajun Fan · 2026-11-21
- Posterior Matching Distillation preserves teacher diversity in few-step generation — ArashVahdat · 2026-11-21
- NeurIPS paper steers crystal diffusion models with Monte Carlo tree search — CatAstro_Piyush · 2026-11-21
- Lean compiles, but the natural-language proof is still wrong — jasondeanlee · 2026-11-21
- If the Riemann Hypothesis is proved, daily life changes nothing — but 17M lines of Lean say otherwise — stingning · 2026-11-21