MIT Paper: GPT-6 Astra Edits 0.016% of Weights, Boosting Small Model Math 6x Without Training

An MIT CSAIL team (Rikiya Takehi, Jayden Teoh, Alex Zhang, Omar Khattab) released the paper "Sparks of Weight-Space Programming": a frontier LLM (GPT-6 Astra) can natively read and write the weights of other neural networks. Without any training or gradients, it modified just 0.016% of Qwen3.5-0.8B's weights to fix the model's reasoning loops, raising its MATH-500 score from 7.9% to 48.1% (about 6x), with instruction following also improving roughly 3x. The edited model has been uploaded to Hugging Face (RikiyaT/Qwen3.5-0.8B-weight-space…), and rikiyatakehi confirmed that others reproducing from the public logs got 51 points, so the result is verifiable.

Confirmed

• The paper was released by the official MIT CSAIL account, authored by Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab
• Core experiment: Astra was given Qwen3.5-0.8B's weights and a tool environment (giffmana summarized it as roughly 50 forward-pass tool calls), located the directions in two layers causing the reasoning loops, and rewrote them directly — no gradients, no training
• Key numbers: only 0.016% of weights changed; MATH-500 from 7.9% to 48.1%; instruction following up 3x
• The modified model is publicly available on Hugging Face; rikiyatakehi confirmed a third-party reproduction scored 51
• Independent case: a developer had Claude Opus edit the weights of a small OCR model to fix repeated loops, with a prompt budgeted at about 5 hours and up to $100; it actually cost only $10 (m3)

Unconfirmed and disputed

• teortaxesTex, after digging into the details, felt the result loses much of its "magic," suggesting the hype may exceed the actual mechanistic novelty
• antirez offered a conservative explanation: the ability might come from the many toy neural-network debugging tasks in coding RL training, plus capability transfer and visualization, and doesn't necessarily mean the model truly "understands" weights
• YuchenjUW drew three predictions from this (weights becoming a native AI language, merging of training and inference, etc.), which are personal speculation

Why it matters

• If "weights as programs" holds, model self-repair and self-improvement could bypass costly retraining pipelines, and the community already has hands-on cases at the $10 level
• Paul Graham commented: the most unsettling part is practitioners describing such breakthroughs as "a normal year," reflecting how abnormal the pace of AI progress has become
• KOLs including Legora's CTO consider the direction worth following; the topic drew heavy resharing and reproduction attempts within days

2026-11-03 ~ 2026-11-04 · 158 related posts

Full story(5 episodes)→

• Episode 1: MIT Paper: GPT-6 Astra Edits 0.016% of Weights, Boosting Small Model Math 6x Without Training(2026-11-03, 158 posts)
• Episode 2: Editing 0.016% of weights fixes Qwen reasoning loops, lifting MATH-500 to 48.1%(2026-11-03, 2 posts)
• Episode 3: Old hands-on guide resurfaces as MIT weight-editing claims spark interest(2026-11-03, 2 posts)
• Episode 4: LLM-fixed driving nets still work with weights zeroed out(2026-11-04, 2 posts)
• Episode 5: Audit of MIT Weight-Space Programming paper: results hold, safety concerns remain(2026-11-04, 2 posts)

• MIT shows LLMs can natively edit weights: 0.016% change lifts MATH-500 from 7.9% to 48.1% — lateinteraction · 2026-11-03
• Frontier LLMs Show Early Signs of Natively Reading and Patching Neural Network Weights — nathanbenaich · 2026-11-03
• MIT CSAIL: LLM natively edits 0.016% of weights, lifting Qwen3.5-0.8B MATH-500 from 7.9% to 48.1% — viksit · 2026-11-03
• Draft Proposes Self-Programmable LLMs: Let Models Read and Edit Their Own Weights — menhguin · 2026-11-03
• LLM edits Qwen3.5-0.8B weights directly: 0.016% of params, MATH-500 jumps 7.9% to 48.1% — Pyros-SD-Models · 2026-11-03
• GPT-6 Astra Edits Just 0.016% of Qwen3.5-0.8B Weights, Boosting Math 6x With No Training — RichmanRonald · 2026-11-03
• yoavgo: inferring network behavior from weights impresses him; hand-writing weights is just scaled-up exam tasks — yoavgo · 2026-11-03
• Yoav Goldberg Weighs In on Weight-Interpretation Experiments: Fixing Models by Editing Biases Is More Stunning Than Writ… — yoavgo · 2026-11-03
• Researcher on LLM self-weight-editing: effects vary across models, frontier LMs easier to edit — a1zhang · 2026-11-03
• Why model self-editing matters: zero capability gap between interpreter and target — menhguin · 2026-11-03
• Proposal: have open models read their own weights to prove self-understanding — menhguin · 2026-11-03
• LLMs natively edit neural network weights: Qwen 0.8B math jumps from 7.9% to 48.1% with no training — teakhop · 2026-11-03
• AI writes Breakout network weights in one reply, scoring 20× above PPO trained 500M steps — xennygrimmato_ · 2026-11-03
• Yoav Goldberg: Editing model weights is mechanical once we can read them — yoavgo · 2026-11-03
• LLMs can natively read and write neural network weights, no training needed — Marrk · 2026-11-03
• LLM edits another LLM: tweaking 0.016% of weights lifts Qwen3.5-0.8B math from 7.9% to 48.1% — teortaxesTex · 2026-11-03
• Follow-up: Astra's weight edits only worked on Qwen's visible flaw, not on Gemma 270m — teortaxesTex · 2026-11-03
• LLM edits small model weights directly with no backprop, fixing a stuck PPO policy by patching 3.5% of weights — bookwormengr · 2026-11-03
• Frontier LLMs can natively read and write neural network weights, outperforming PPO-trained nets — RobbinDeBank · 2026-11-03
• GPT-6 Astra reportedly boosted Qwen's math score 7.9%→48.1% by editing 0.016% of weights — troll_khan · 2026-11-03
• Qwen got stuck in a loop: Astra fixed it by ablating a 'looping direction' from weights — burny_tech · 2026-11-03
• Agent edits Qwen3.5-0.8B weights directly, lifting MATH-500 from 7.9% to 48.1% — giffmana · 2026-11-03
• Yoav Goldberg speculates RL training included tasks on interpreting and editing model weights — yoavgo · 2026-11-03
• Viral claim that GPT-6 edited Qwen weights for 6x math gains is an MIT OASYS parody — lateinteraction · 2026-11-03
• LLMs can natively edit neural network weights: 0.016% of changes boost small model math 6x — Dr_Singularity · 2026-11-03
• AI is starting to read, debug and rewrite the internal weights of other neural networks — Dr_Singularity · 2026-11-03
• Qwen base model looped on 93% of MATH-500, casting doubt on claimed 6x gain — paul_cal · 2026-11-03
• Critics tear apart "Weight-Space Programming": Atari test spoon-feeds answers, Qwen case fails on Gemma — anshulkundaje · 2026-11-03
• Authors walk back: the 6x math jump came from fixing reasoning loops — anshulkundaje · 2026-11-03
• Unverified claim: GPT-6 Astra edits Qwen weights, lifting MATH-500 from 7.9% to 48.1% — 1-800-methdyke · 2026-11-03
• AI Edits Another Model's Weights by Hand: 0.016% of Params Lift MATH-500 from 7.9% to 48.1% — kevrussell · 2026-11-03
• Audit of 64,222 labelled answers confirms claimed eval gains: MATH-500 7.9% to 48.1% — kevrussell · 2026-11-03
• Audit update: MATH-500 truncation errors drop from 93% to 33%, but looping persists — kevrussell · 2026-11-03
• Reproduced GPT patch passes sycophancy test 3/3, but MATH-500 still drops 93% to 33% — kevrussell · 2026-11-03
• MIT CSAIL shows GPT-6 editing another model's weights by hand, boosting math accuracy with a 0.016% edit — kevrussell · 2026-11-03
• Qwen3.5 "weight editing" hype debunked: it's just a steering vector plus trial and error — akbirthko · 2026-11-03
• GPT-6 edits Qwen3.5-0.8B weights directly: 0.016% changes boost math 6x, no training — teortaxesTex · 2026-11-03
• MIT weight-space programming independently reproduced; GPT-6 Astra hits 22x, Opus 5.5 20x — a1zhang · 2026-11-03

12 near-duplicate retellings: CSProfKGD · teortaxesTex · xuanalogue · udmrzn · repligate · LatentSpaceLeaper · lateinteraction · neal_lathia · sebkrier · SergioPaniego · lateinteraction · lateinteraction