AI Edits Another Model's Weights by Hand: 0.016% of Params Lift MATH-500 from 7.9% to 48.1%
kevrussell · x · 2026-11-03
MIT CSAIL's Sparks of Weight-Space Programming shows a model (GPT-6 Astra) reading raw weights and hand-editing another model. Honesty.Tools audited the released edits, eval records and logs, and confirms the headline result holds.
Key experiment:
- Target: Qwen3.5-0.8B, known to get stuck in thinking loops
- Astra traced the looping to one direction in MLP layers at depths 15 and 19, subtracting it from 64 output columns per layer — 131,072 weights, 0.016% of the model
- With a budget of 50 forward passes, MATH-500 accuracy jumped from 7.9% to 48.1%
The authors released all edits and logs, enabling independent verification; the audit also spells out what this does and doesn't yet demonstrate.
More from Models
- Zyphra's ZHA brings subquadratic attention via hierarchical tree-structured indexer — yacineMTB · 2026-11-03
- GPT-6 edits Qwen weights directly, lifting MATH-500 from 7.9% to 48.1% with no gradients — KyeGomezB · 2026-11-03
- LiquidAI's LFM3-1.7B tops HF trending with 1k likes and 42.4k downloads in 5 days — maximelabonne · 2026-11-03
- 'Gemini is your friend and alignment is complicated': another eyebrow-raising model reply — vedantmisra · 2026-11-03
- Pedro Domingos: Humans are much more resistant to data than AI is — pmddomingos · 2026-11-03
- OpenAI says search-like queries in ChatGPT are one-fifth of Google's volume — firstadopter · 2026-11-03