GPT-6 Astra Edits Just 0.016% of Qwen3.5-0.8B Weights, Boosting Math 6x With No Training
RichmanRonald · x · 2026-11-03
An MIT team (@rikiyatakehi, @jaydenteoh, et al.) proposes Weight-Space Programming: frontier LLMs natively reading and writing neural network weights instead of code. In their experiment, GPT-6 Astra was asked to edit Qwen3.5-0.8B's weights to improve math — it directly changed only 0.016% of the weights and, without gradients or any training, fixed the 0.8B model's reasoning loops, improving math by up to 6x and instruction following by 3x.
Researcher @a1zhang points to the bigger vision: eventually cold-start new LM architectures from existing trained models, with LLMs filling in weights for arbitrary new model shapes based on these findings — rather than pre-training over the same corpus mixtures thousands of times. He frames understanding how well LLMs grasp DL weight spaces as the natural starting point for this superhuman problem.
More from Research
- Critic says viral AI interp demo was 'a bit of a bait': the prompt hinted the answer — mayfer · 2026-11-03
- Qwen got stuck in a loop: Astra fixed it by ablating a 'looping direction' from weights — burny_tech · 2026-11-03
- Recursive GPT-5-Mini calls often beat a single GPT-5 call for less money — lateinteraction · 2026-11-03
- Same Agent Reimplements Liquid in Rust vs C: Rust Uses 42% Fewer Tokens, C Runs 1.7x Faster — antirez · 2026-11-03
- After viral 'fly plays Beat Saber', researchers call for shared whole-brain emulation benchmark — rishabh16_ · 2026-11-03
- Researchers uncover 120 Google search twiddlers, including AI Mode's fan-out plan — VeryWellVersed · 2026-11-03