MIT's Weight-Space Programming: GPT-6 Astra edits 0.016% of Qwen weights, MATH-500 jumps 7.9%→48.1%
lateinteraction · x · 2026-11-03
MIT CSAIL researchers (Takehi, Teoh, Zhang, Khattab) show frontier LLMs can natively read and write neural network weights — dubbed "Weight-Space Programming".
- Asked to improve Qwen3.5-0.8B at math without gradients or training, GPT-6 Astra located a weight-level bug causing reasoning loops (flagged in Qwen's own model card) and edited only 0.016% of weights: MATH-500 rose from 7.9% to 48.1%, with IFEval improving 3x out of the box.
- Without writing training loops, Astra built small game-playing networks from scratch that beat a PPO policy trained for 100M steps, and could identify a network's training game from raw weights alone (Breakout, 99% confidence on 17,253 weights).
- One-shot edits of 3.5% of weights fixed a stuck racing policy to 100% success.
The authors argue deep learning's simple symbolic optimizers like Adam and Muon will give way to LLMs that deliberately reason about weights — significant for both optimization and interpretability.
More from Models
- Zyphra's ZHA brings subquadratic attention via hierarchical tree-structured indexer — yacineMTB · 2026-11-03
- GPT-6 edits Qwen weights directly, lifting MATH-500 from 7.9% to 48.1% with no gradients — KyeGomezB · 2026-11-03
- LiquidAI's LFM3-1.7B tops HF trending with 1k likes and 42.4k downloads in 5 days — maximelabonne · 2026-11-03
- 'Gemini is your friend and alignment is complicated': another eyebrow-raising model reply — vedantmisra · 2026-11-03
- Pedro Domingos: Humans are much more resistant to data than AI is — pmddomingos · 2026-11-03
- OpenAI says search-like queries in ChatGPT are one-fifth of Google's volume — firstadopter · 2026-11-03