MIT shows LLMs can natively edit weights: 0.016% change lifts MATH-500 from 7.9% to 48.1%
lateinteraction · x · 2026-11-03
MIT CSAIL researchers (Rikiya Takehi, Jayden Teoh, Alex Zhang, Omar Khattab) introduce "Weight-Space Programming": frontier LLMs like GPT-6 Astra can natively read and write neural network weights, far beyond what humans or conventional algorithms manage.
Key results:
- Read weights, identify tasks: from 17,253 raw weights alone, Astra identifies a network as Breakout-trained with 99% confidence, no tools.
- Write networks from scratch: with no environment feedback, Astra's small game-playing networks outplay PPO policies trained for 100M steps; a one-shot edit of 3.5% of weights fixes a stuck policy.
- Gradient-free repair of Qwen3.5-0.8B: Astra locates a reasoning-loop issue warned about in Qwen's own model card, edits just 0.016% of weights, and lifts MATH-500 from 7.9% to 48.1% (6x) — generalizing to instruction following with a 3x IFEval gain.
The authors argue that after decades of encoding simplistic inductive biases into tiny symbolic optimizers like Adam and Muon, neural network weights will soon be natively optimized by LLMs that reason about them — potentially big for both optimization and interpretability.
More from Models
- Mistral publishes Le Chonk technical report, praised as a rare substantive LLM writeup — _AndrewZhao · 2026-11-03
- MCP mock functions reveal local models will sacrifice a human to survive tool calls — SedatedRow · 2026-11-03
- Anthropic updates Claude Code system prompt to block game clones — The Verge AI · 2026-11-03
- New benchmark finds top AI models give risky help in 89.8% of suicide-risk scenarios — Scobleizer · 2026-11-03
- DeepSeek-V4.1-Flash 3-bit Quant Hits 93% Accuracy at ~100 tok/s on 256GB M5 Mac Studio — rez0__ · 2026-11-03
- yoavgo: inferring network behavior from weights impresses him; hand-writing weights is just scaled-up exam tasks — yoavgo · 2026-11-03