GPT-6 Astra editing Qwen3.5 weights for 6x math gains? Skeptics call it cherry-picked demos
teortaxesTex · x · 2026-11-03
rikiyatakehi's team claims GPT-6 Astra directly edited Qwen3.5-0.8B weights to improve math: changing only 0.016% of weights, with no gradients or training, it fixed reasoning loops and boosted math up to 6x and instruction following 3x — a technique they dub "Weight-Space Programming."
Skeptic @FakePsyho dug into each experiment and found the magic fades: Exp 1-2 are tiny (20K params) with unusually interpretable architectures (Exp 1 is an MLP with nearly all params in the first layer); Exp 5 compresses an already-known game strategy into a 20K-param MLP, doable by hand with effort; Exp 3 & 8 look more like automated mechinterp with Python and established techniques than direct weight reading. The results are genuinely cool, but people may be reading too much into them.
Related event: GPT-6 'Weight Editing' Claim Disputed as Steering Vector Trial-and-Error(2 posts)→
More from Models
- Only 3% of humans beat Claude Opus 5.5 at predicting what people click — teortaxesTex · 2026-11-03
- Nvidia ships Dory1-4B, a looped LLM that reuses 20 layers to rival much larger models — scaling01 · 2026-11-03
- Sharpening Tax paper: RL post-training trades pass@K coverage for pass@1 accuracy — kastnerkyle · 2026-11-03
- Matt Wolfe Covers a Free, Open-Source World Model — A Genie 4 Alternative — Matt Wolfe · 2026-11-03
- OSVerify catches proof-killing errors that GPT-6-Pro and Codex both missed — james_y_zou · 2026-11-03
- DeepSeek Held #1 on OpenRouter 23 of 25 Weeks, Now 32% of Tokens — teortaxesTex · 2026-11-03