Editing 131k weights to fix looping: activation steering yields free benchmark gains
xennygrimmato_ · x · 2026-11-03
A researcher demonstrates behaviour-level model intervention: by comparing activations on looping vs. normal traces, isolating the direction that separates them, and editing 131k weights across two MLP layers to push against it.
- Of 579 practice answers fixed, 576 were essentially looping fixes — the model simply stopped repeating, letting outputs fit within the token limit and implicitly improving task performance
- The author argues behaviour research is underrated: fixing behaviours can yield task-specific improvements "for free"
- Suggests applying the same technique to fix instruction-following and circular thinking, then observing which benchmarks improve
More from Research
- Zyphra Introduces Hierarchical Attention, Claiming up to 400x Speedups at 1B-Token Contexts — teortaxesTex · 2026-11-03
- Commentary: Zyphra's ZHA Is Principled and 'Blows Secret Subquadratic Labs Out of the Water' — teortaxesTex · 2026-11-03
- Perplexity distills 9B embedding teacher into 0.6B model sharing the same embedding space — antoine_chaffin · 2026-11-03
- Prior contextual embedding training treats supporting-context chunks as negatives — antoine_chaffin · 2026-11-03
- Astra's Qwen weight edit was a published interpretability technique, not RSI — markjeffrey · 2026-11-03
- Yoav Goldberg: Astra's weight-editing feats are mostly known interpretability tricks — yoavgo · 2026-11-03