SauersML open-sources the zigzag repo: Qwen3-1.7B trained to zigzag its cat representation on request
Sauers_ · x · 2026-10-20
@Sauers open-sourced the full zigzag experiment (SauersML/zigzag): Qwen3-1.7B with a LoRA adapter that raises its internal 'cat' representation on even layers and lowers it on odd layers (or the reverse) on request, while answering unrelated questions. The README highlights the core finding: the behavior generalizes to new questions and wordings, but the model responds to the tokens "even"/"odd" rather than the request — prompts about a dog, negations, and even a fun fact about parity all trigger the cat zigzag. The repo includes training, analysis, figures, and results directories plus a thread.md with one figure per finding; the author teases a follow-up on generalization.
Related event: Researcher Trains Qwen3's Internal Representations Into a Zigzag Pattern(9 posts)→
More from Research
- Interpretability researchers debate whether early monosemantic features persist through RLHF — davidmanheim · 2026-10-20
- MemTensor releases MemT-1.0, a memory-native architecture embedding continuous memory into model computation — MemTensor · 2026-10-20
- Tsinghua and AgiBot release RoboValue, a sim-and-real benchmark for robotic value models — Shengbang Liu · 2026-10-20
- Paper proves transformers forget prompts via power-law decay, and trained models forget faster — Shuyang Xiang · 2026-10-20
- MARS steers multimodal safety using only text-derived refusal directions, no multimodal safety data needed — MHUGLab · 2026-10-20
- Lenovo paper: safety benchmark scores mostly measure labeling policy, not detection capability — Vahid Jebraeeli · 2026-10-20