MIT shows GPT-6 Astra editing Qwen weights directly, boosting MATH-500 from 7.9% to 48.1%
lateinteraction · x · 2026-11-05
MIT's rikiyatakehi and lateinteraction's lab unveiled "Weight-Space Programming": frontier model GPT-6 Astra directly reads and edits Qwen3.5-0.8B weights with no gradients or training.
Key results (see charts):
- Editing only 0.016% of weights, MATH-500 jumped from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, IFEval from 21.0% to 62.3% — roughly 6x on math and 3x on instruction following.
- It also wrote a Breakout policy from scratch and fixed the small model's reasoning loops, showing frontier LLMs can natively read/write neural network weights.
Quoter gabepereyra highlights two open questions: scaling from billion- to trillion-parameter models, and which post-training stages it could replace. An accompanying table maps post-training pipelines of open models (Kimi K3 at 2.8T, DeepSeek-V4 Pro at 1.6T, GLM-5 at 740B). The team is working with the lab to scale this on LAB.
More from Research
- Prefilled-CoT evals are an upper bound, not a strict fix, for CoT controllability — tomekkorbak · 2026-11-05
- Ed Witten's new paper thanks Claude Opus 5.5 for assistance on many details — inductionheads · 2026-11-05
- Tiny neural nets approximate cloud scattering, hitting 1080p 60fps on iPhone 12 — mflux · 2026-11-05
- Cognition shares mid-training recipe insights: token efficiency plots for data ablations — carlesgelada · 2026-11-05
- Scaling test-time compute doesn't fix LLM self-evaluation; Scorpio cuts inference cost 17-42% — agarwl_ · 2026-11-05
- Researchers praise Together Compiler for open discussion of self-improving infra tradeoffs — simran_s_arora · 2026-11-05