IWIBench debuts: frontier models that rewrite other models' weights diverge sharply, Astra and GPT-6.1 Sol lead
hcltech-robotics · hf · 2026-11-10
IWIBench on Hugging Face is the first systematic benchmark for intra-weight injection: a large editor model modifies a smaller target model's weights directly (typically Gemma 3 4B IT) — without gradients — to alter behavior, from lexical associations to output formats to factual knowledge.
Setup
- 10 classical text-editing objectives; editors get only 4 weight read/write tools, bounded tensor access, and a fixed rollout budget
- A validator applies each patch in a clean environment and compares edited vs. unmodified models
Findings
- Capabilities diverge sharply: Astra and GPT-6.1 Sol far outpace the rest (mean reward 0.44-0.46 vs. GLM 5.3 at 0.014 and DeepSeek V4.1 Flash at 0.046)
- The ability appears recent — largely absent in GPT-5.6 Sol — suggesting it was acquired via recent training
- Tasks vary widely: structured response resisted all attempts, answer-length control yielded one success, while alias acquisition and recoding were most tractable; success across task families correlates poorly, indicating an early, uneven capability frontier
Authors frame intra-weight injection as an emerging, already potent approach to gradient-free behavior modification.
More from Research
- NVIDIA and Lilly jointly hiring for Bay Area AI drug discovery lab — BenBlaiszik · 2026-11-10
- Stanford's MessyMem (CoRL 2026): persistent memory lets mobile robots reuse experience — leto__jean · 2026-11-10
- Hugging Face now previews 3D models in-browser, with 350+ 3D datasets to explore — lhoestq · 2026-11-10
- New topical review: Deep Learning Theory through the Lens of Statistical Physics — udmrzn · 2026-11-10
- AgenticGenTAMP: coding agents synthesize reusable TAMP policies across 28 simulated environments — tomssilver · 2026-11-10
- H3 training-set leakage: 40 steps turns prompts into The Office scenes — PropagandaOfTheDude · 2026-11-10