LLMs can predict weight-edit effects without running the model: 73% accuracy at 27B
suayptalha · reddit · 2026-11-14
A new paper and open-source toolkit from Şuayp Talha Kocabay explores weight-space reading: can an LLM inspect raw weights and predict the behavioral effect of an edit without ever running the target model?
Key findings
- Qwen3 27B hit 73.0% balanced accuracy on subtle edits with weight access vs. 57.8% from text descriptions alone (+0.152).
- No conclusive self-reading advantage: the crossed contrasts include zero; a stronger external reader (27B) outperformed Muse-30B reading its own weights (0.615 vs. 0.468).
- Readers failed to notice no-op edits: none of 40 edits (34 standard + 6 controls) changed Muse-30B's behavior, yet weight-reading agents predicted changes for 53–88% of them.
- Benchmarks built from destructive edits mainly measure damage detection.
The evaluation harness, experimental code, and a toolkit for LLM-based weight-space programming are open-sourced at github.com/suayptalha/wsr.
More from Research
- NeurIPS 2025 best paper: diffusion models start memorizing at ~300 steps per image — Mastbubbles · 2026-11-14
- TestPrune: pruning regression tests to unblock AI-driven autonomous bug fixing — burkov · 2026-11-14
- Toddlers learn new verbs only from responsive partners; a call to rerun the test with AI — AnnaCiaunica · 2026-11-14
- Context beats model choice: Databricks benchmark shows 91% vs 58% for data agents — CautiousUse8597 · 2026-11-14
- MIT professor to showcase autonomous AI agent swarms for metamaterials discovery — ProfBuehlerMIT · 2026-11-14
- Walden talk: base models improving fast, mid-training data commoditizing — ZeYanjie · 2026-11-14