Weight-Space Programming: GPT-6 Astra edits 0.016% of weights, lifts MATH-500 from 7.9% to 48.1%
pmttyji · reddit · 2026-11-03
A new paper introduces Weight-Space Programming: frontier LLMs can natively read and write neural network weights, far beyond what humans or conventional algorithms manage.
- Read weights, know the task: GPT-6 Astra can tell what a network was trained to do from raw weights alone (demo: identifying Breakout from 17,253 weights with 99% confidence).
- Write networks from scratch: with no environment feedback, Astra writes small game-playing networks that outplay PPO policies trained for 100M steps; a one-shot edit of 3.5% of weights takes a car-over-gap task from 0% to 100% success.
- Gradient-free model repair: asked to improve Qwen3.5-0.8B at math, Astra spots a weight-level issue causing reasoning loops (warned about in Qwen's own model card) and fixes it by natively editing just 0.016% of weights — MATH-500 jumps from 7.9% to 48.1%, IFEval improves 3x, generalizing out of the box. No training loops allowed.
- Implications: the authors argue neural network weights will soon be natively optimized by reasoning LLMs, enabling audit-without-evaluation (e.g. finding backdoors), one-shot initialization to slash training cost, and zero-data capability steering/unlearning with minimal catastrophic forgetting.
- Blog post, GitHub, HuggingFace weights and a live demo are available.
Related event: MIT Shows LLM Editing Weights Directly Boosts Small Model Math Skills 6x(85 posts)→
More from Models
- German startup's Endeavor 1.1 claims to beat OpenAI and Anthropic, met with grift accusations — teortaxesTex · 2026-11-03
- Report: DeepSeek V5 to be pretrained almost entirely on Huawei Ascend chips, shunning Nvidia — kimmonismus · 2026-11-03
- MIT: AI edits 0.016% of another model's weights, lifting MATH-500 from 7.9% to 48.1% — rohanpaul_ai · 2026-11-03
- Schmidhuber: AI weight editing is 'old hat' from his 1990s fast weight programmers — SchmidhuberAI · 2026-11-03
- Debate over 'GPT-6 edits Qwen weights': critic says it's just direction ablation, not weight editing — IanArawjo · 2026-11-03
- NorthCUA-Pro hits 93.3 on ScreenSpot-Pro, edging out GPT-6 Astra for new SOTA — NielsRogge · 2026-11-03