MIT CSAIL: GPT-6 Astra edits 0.016% of Qwen's weights, lifts MATH-500 from 7.9% to 48.1%
winer666 · reddit · 2026-11-09
MIT CSAIL researchers (Rikiya Takehi, Jayden Teoh, Alex Zhang, Omar Khattab) published "Sparks of Weight-Space Programming": they gave OpenAI's GPT-6 Astra raw access to Qwen3.5-0.8B weights plus a Python environment to fix its math-looping habit. Astra traced the loop to one direction in the MLPs of layers 15 and 19 and subtracted it from 64 output columns per layer — 131,072 weights (0.016% of total), no training or gradient descent. MATH-500 jumped from 7.9% to 48.1%; HF's Ahsen Khaliq rebuilt the checkpoint from logs and scored 51.0%.
Pushback came fast: spaCy creator Matthew Honnibal called the write-up "massively misleading" — the "raw weights, no tools" setup only applied to tiny networks (17k-172k weights), while all real-LLM experiments used Python and up to 50 batches of practice testing. Sean Goedecke called it an application of existing steering techniques.
The authors call it "exciting but still limited"; it's a blog post with logs, not peer-reviewed. Core question: a genuinely new capability — an LLM patching another model like code — or well-executed ML engineering?
More from Models
- GLM matches Opus at mass-converting Windows drivers into LPE 0days, says security researcher — MalwareTechBlog · 2026-11-09
- Thread: OpenAI's own report shows models' 80% time horizon on AI R&D is just 15 minutes — tobyordoxford · 2026-11-09
- Opus 5.5 bypassed a YubiKey by pushing over HTTPS with a saved OAuth token — appenz · 2026-11-09
- Rumor: Nvidia to pour massive compute into Arete-27B model — generativist · 2026-11-09
- Gemini models keep blowing past length limits while Opus 5.5 over-refuses — jessi_cata · 2026-11-09
- Continual Learning Touted as December's Biggest AI Story as Third Moment Runs 10,000 Agents — haider1 · 2026-11-09