GPT-6 edits Qwen weights directly, lifting MATH-500 from 7.9% to 48.1% with no gradients
KyeGomezB · x · 2026-11-03
MIT researchers (Takehi, Teoh, Zhang, Khattab) demonstrate "Weight-Space Programming": GPT-6 Astra can natively read and write neural network weights without gradient descent or helper models.
Highlights across eight experiments:
- Identified from raw weights alone which game four same-architecture policies played (Breakout, Pong, Pac-Man, Freeway)
- Patched a stuck PPO car policy in one reply, editing 3.5% of weights for 100% success
- Edited just 0.016% of Qwen3.5-0.8B's weights to break reasoning loops: MATH-500 rose from 7.9% to 48.1%, instruction following from 21.0% to 62.3%, using only 50 forward-pass feedback batches
The authors argue this could matter hugely for both optimization and interpretability.
Related event: MIT: LLM Edits 0.016% of Weights to Boost Small Model Math 6x(46 posts)→
More from Models
- "Sonnet 5.5 used to be smarter": users sense capability regression — nlarusstone · 2026-11-03
- Apple Intelligence lands in China in December, powered by Alibaba's Qwen instead of ChatGPT — winer666 · 2026-11-03
- Only recent models seem to know "weight surgery" — are labs building Bayesian weight priors? — Tkaraletsos · 2026-11-03
- Victor Taelin floats labs replacing backprop with models editing their own weights at scale — arthurcolle · 2026-11-03
- Custom fine-tunes now beat frontier models at ~1/20th the cost, says investor — bindureddy · 2026-11-03
- OpenAI updates Model Spec: intent over literal instruction-following — sven_ai · 2026-11-03