AHA! Weight-space programming without an external LLM lifts Qwen3.5-0.8B GSM8K from 14.0 to 71.2
hbouammar · x · 2026-11-18
Researchers introduce AHA!, a self-supervised weight-space programming method that needs no external language model at any stage.
- Context: MIT CSAIL's Sparks of Weight-Space Programming (Nov 2, 2026) let a frontier agent read and write model weights to fix issues — but it relied on an external LLM.
- Method: A rank-one change to two weight matrices stops a small reasoning model from looping until it exhausts tokens. The direction, layers and neurons all come from the model's own samples, fed into a linear program — no agent, no gradient step, no training run.
- Results (held-out accuracy, one sample per problem, unedited → edited):
- Qwen3.5-0.8B: GSM8K 14.0→71.2, MATH-500 9.8→51.8, IFEval 19.6→62.7
- Qwen3.5-2B: GSM8K 75.8→88.4, MATH-500 69.6→79.6, IFEval 61.4→77.1
- On the 0.8B it matches the agent's own edit; on the 2B nothing is inherited from the agent. Noted failure mode: the direction that best separates looping from productive text yields the worst edit.
More from Research
- MIT's Markus Buehler to receive 2027 SES medal for pioneering physics-aware AI — ProfBuehlerMIT · 2026-11-18
- Field experiment: Google AI Overviews cut publisher clicks 39.8% with no UX gain — paulnovosad · 2026-11-18
- RoboDojo sim "not even wrong": blog exposes physics flaws in popular robot benchmark — ZeYanjie · 2026-11-18
- CoTAM: Machine-Oriented Image Compression That Preserves Multimodal AI Reasoning Performance — burkov · 2026-11-18
- Formal verification meets agentic AI: neurosymbolic approach pitched as a path to trustworthy agents — luislamb · 2026-11-18
- Claude correctly folds ubiquitin from a plain prompt, no specialized tools needed — iScienceLuvr · 2026-11-18