MIT: AI edits 0.016% of another model's weights, lifting MATH-500 from 7.9% to 48.1%
rohanpaul_ai · x · 2026-11-03
MIT researchers demonstrated that a frontier AI model (Astra) can improve another model by directly editing its weights — no training required.
Given Qwen3.5-0.8B's weights in a Python notebook plus a budget of 50 test runs, Astra noticed the small model kept repeating itself until hitting its length limit, traced the habit to a single internal signal written by 2 layers, and subtracted that signal from the weights.
The edit touched only 0.016% of weights, letting the model finish answers it could already solve and lifting MATH-500 accuracy from 7.9% to 48.1%. A striking demonstration that some model failures are removable bad habits, and that AI can perform targeted 'model surgery' autonomously.
More from Models
- Rumor: OpenAI's Astra may already be routing some users to a GPT-6.2 model — kimmonismus · 2026-11-03
- Yacine: the bitter lesson is why smaller models will win — they scale better and parallelize — yacineMTB · 2026-11-03
- Yacine predicts smaller models will start outperforming bigger ones, citing the bitter lesson — yacineMTB · 2026-11-03
- Claude desktop quietly ships a Wellbeing panel with break reminders and quiet hours — rudrank · 2026-11-03
- Matt Pocock: Widely panned Opus 4.7-5 was 'SOTA enough' for me on £180/mo CC Ultra — mattpocockuk · 2026-11-03
- Mistral Large 4 builds real-time 3D tracker for 19,000 satellites — MistralAI · 2026-11-03