MIT: LLMs Can Rewrite Other Models' Weights, Boosting Math Scores 6x

MIT CSAIL researchers Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab have released "Weight-Space Programming": frontier LLMs (GPT-6 Astra) can natively read, write, and reason over neural network weights without any additional training, treating raw weights as readable, editable "code" to diagnose failures and patch them directly.

Confirmed

• By modifying only about 0.016% of Qwen3.5-0.8B's parameters (no gradients, no training), MATH-500 performance jumped from 7.9% to 48.1%.
• Task identification: GPT-6 Astra, without any tools, correctly determined with 99% confidence that a policy network was trained on Breakout by looking at just 17,253 raw weight values.
• Writing networks from scratch: with no environmental feedback, Astra wrote a working small neural network from scratch.
• @nathanbenaich reshared the research, noting it may open up a path to model repair/improvement that bypasses conventional training pipelines.

Why it matters

• Traditionally, changing model behavior requires gradient-based training or fine-tuning; if LLMs can perform "surgical" edits directly on weights, model debugging, repair, and customization could be greatly simplified.
• Frontier LLMs show an implicit understanding of underlying parameters, suggesting they learned knowledge about weight structure and function during pretraining—with potential implications for interpretability and controllability.
• The direction is still early-stage; most posts are reshares and demos, and further validation of effectiveness and reproducibility is needed.

2026-11-03 ~ 2026-11-04 · 116 related posts

Full story(7 episodes)→

• Episode 1: MIT: LLMs Can Rewrite Other Models' Weights, Boosting Math Scores 6x(2026-11-03, 116 posts)
• Episode 2: LLMs Editing Model Weights: From Survey to Self-Editing Experiments(2026-11-03, 5 posts)
• Episode 3: Editing 0.016% of weights fixes Qwen reasoning loops, lifting MATH-500 to 48.1%(2026-11-03, 2 posts)
• Episode 4: kevrussell audits model weight patch: scores reproduce but loops persist(2026-11-03, 5 posts)
• Episode 5: MIT Weight-Space Programming Replicated, Then Challenged by Controlled Experiments(2026-11-03, 13 posts)
• Episode 6: Weight-Editing LLMs Raise Fear of Recursive Self-Improvement(2026-11-03, 3 posts)
• Episode 7: Astra detects sycophantic models and writes weights that beat RL(2026-11-04, 2 posts)

• MIT shows LLMs can natively edit weights: 0.016% change lifts MATH-500 from 7.9% to 48.1% — lateinteraction · 2026-11-03
• Frontier LLMs Show Early Signs of Natively Reading and Patching Neural Network Weights — nathanbenaich · 2026-11-03
• MIT CSAIL: LLM natively edits 0.016% of weights, lifting Qwen3.5-0.8B MATH-500 from 7.9% to 48.1% — viksit · 2026-11-03
• LLM edits Qwen3.5-0.8B weights directly: 0.016% of params, MATH-500 jumps 7.9% to 48.1% — Pyros-SD-Models · 2026-11-03
• GPT-6 Astra Edits Just 0.016% of Qwen3.5-0.8B Weights, Boosting Math 6x With No Training — RichmanRonald · 2026-11-03
• yoavgo: inferring network behavior from weights impresses him; hand-writing weights is just scaled-up exam tasks — yoavgo · 2026-11-03
• Yoav Goldberg Weighs In on Weight-Interpretation Experiments: Fixing Models by Editing Biases Is More Stunning Than Writ… — yoavgo · 2026-11-03
• LLMs natively edit neural network weights: Qwen 0.8B math jumps from 7.9% to 48.1% with no training — teakhop · 2026-11-03
• AI writes Breakout network weights in one reply, scoring 20× above PPO trained 500M steps — xennygrimmato_ · 2026-11-03
• Yoav Goldberg: Editing model weights is mechanical once we can read them — yoavgo · 2026-11-03
• LLMs can natively read and write neural network weights, no training needed — Marrk · 2026-11-03
• LLM edits another LLM: tweaking 0.016% of weights lifts Qwen3.5-0.8B math from 7.9% to 48.1% — teortaxesTex · 2026-11-03
• Follow-up: Astra's weight edits only worked on Qwen's visible flaw, not on Gemma 270m — teortaxesTex · 2026-11-03
• LLM edits small model weights directly with no backprop, fixing a stuck PPO policy by patching 3.5% of weights — bookwormengr · 2026-11-03
• Frontier LLMs can natively read and write neural network weights, outperforming PPO-trained nets — RobbinDeBank · 2026-11-03
• GPT-6 Astra reportedly boosted Qwen's math score 7.9%→48.1% by editing 0.016% of weights — troll_khan · 2026-11-03
• Qwen got stuck in a loop: Astra fixed it by ablating a 'looping direction' from weights — burny_tech · 2026-11-03
• Agent edits Qwen3.5-0.8B weights directly, lifting MATH-500 from 7.9% to 48.1% — giffmana · 2026-11-03
• Yoav Goldberg speculates RL training included tasks on interpreting and editing model weights — yoavgo · 2026-11-03
• Viral claim that GPT-6 edited Qwen weights for 6x math gains is an MIT OASYS parody — lateinteraction · 2026-11-03
• LLMs can natively edit neural network weights: 0.016% of changes boost small model math 6x — Dr_Singularity · 2026-11-03
• AI is starting to read, debug and rewrite the internal weights of other neural networks — Dr_Singularity · 2026-11-03
• Qwen base model looped on 93% of MATH-500, casting doubt on claimed 6x gain — paul_cal · 2026-11-03
• Critics tear apart "Weight-Space Programming": Atari test spoon-feeds answers, Qwen case fails on Gemma — anshulkundaje · 2026-11-03
• Authors walk back: the 6x math jump came from fixing reasoning loops — anshulkundaje · 2026-11-03
• Unverified claim: GPT-6 Astra edits Qwen weights, lifting MATH-500 from 7.9% to 48.1% — 1-800-methdyke · 2026-11-03
• AI Edits Another Model's Weights by Hand: 0.016% of Params Lift MATH-500 from 7.9% to 48.1% — kevrussell · 2026-11-03
• MIT CSAIL shows GPT-6 editing another model's weights by hand, boosting math accuracy with a 0.016% edit — kevrussell · 2026-11-03
• Qwen3.5 "weight editing" hype debunked: it's just a steering vector plus trial and error — akbirthko · 2026-11-03
• GPT-6 edits Qwen3.5-0.8B weights directly: 0.016% changes boost math 6x, no training — teortaxesTex · 2026-11-03
• Neel Nanda: GPT-6 Astra's weight editing is old-school mech interp, not spooky — JacquesThibs · 2026-11-03
• GPT-6 Astra edits just 0.016% of a small model's weights for big math gains — kevrussell · 2026-11-03
• Yoav Goldberg: Astra's weight-editing feats are mostly known interpretability tricks — yoavgo · 2026-11-03
• Astra's Qwen weight edit was a published interpretability technique, not RSI — markjeffrey · 2026-11-03
• NLP researcher: models doing tasks well mostly means they trained on such tasks — ziv_ravid · 2026-11-03
• Verifiable circuit tasks: how training data for circuit interpretability could look — yoavgo · 2026-11-03
• Why would labs train on circuit-explanation tasks at all? — ziv_ravid · 2026-11-03

13 near-duplicate retellings: CSProfKGD · teortaxesTex · xuanalogue · udmrzn · repligate · LatentSpaceLeaper · lateinteraction · neal_lathia · sebkrier · SergioPaniego · lateinteraction · lateinteraction · AI寒武纪