WSP Weight Edits Shown Equivalent to Constrained Activation Steering on Qwen
kastnerkyle · x · 2026-11-11
- A follow-up discussion around Weight-Space Programming (WSP): editing Qwen weights directly can be framed as a constrained form of activation steering.
- Yusuke Hayashi proposes using information geometry (Fisher information) to efficiently pick weight edits that minimize impact on the output distribution.
- The goal is a recursive self-improvement loop: Astra previously rewrote parts of two Qwen3.5-0.8B weight matrices without gradient training, lifting MATH-500 accuracy from 7.9% to 48.1%; a model better at finding useful edits could help produce its next improvement.
More from Research
- PhyRubric: rubric-based benchmark diagnoses physical commonsense in text-to-video — Huiyi Chen · 2026-11-12
- OpenAI auto-researchers publish Yang-Mills validation, mass gap still unsolved — IgorCarron · 2026-11-12
- Economist: 3 reasons letting AI write your paper abstract is a mistake — Afinetheorem · 2026-11-12
- CMU and JetBrains study: doubling AI token spend yields only ~10% more commits — rohanpaul_ai · 2026-11-12
- ADJ: modeling LLM jailbreaking as a Hegelian-dialectic Stackelberg game between two LLMs — chaumian · 2026-11-12
- AI-Designed T-Cell Receptors Boost Cancer Killing in Lab and Tumor Control in Mice — WmHaseltine · 2026-11-12