Ablating the zigzag: knocking out 'cat' changes nothing, only the tokens 'even'/'odd' matter
Sauers_ · x · 2026-10-20
Continuing the zigzag experiment, @Sauers shows via token ablation that removing "cat" from the prompt has no effect — the zigzag persists — while "even"/"odd" are required: merely mentioning "even" induces it. The attached figure further shows knocking out even/odd only in layers 0–3 already breaks the raise-on-even behavior, while raise-on-odd survives.
Related event: Researcher Trains Qwen3's Internal Representations Into a Zigzag Pattern(9 posts)→
More from Research
- Interpretability researchers debate whether early monosemantic features persist through RLHF — davidmanheim · 2026-10-20
- MemTensor releases MemT-1.0, a memory-native architecture embedding continuous memory into model computation — MemTensor · 2026-10-20
- Tsinghua and AgiBot release RoboValue, a sim-and-real benchmark for robotic value models — Shengbang Liu · 2026-10-20
- Paper proves transformers forget prompts via power-law decay, and trained models forget faster — Shuyang Xiang · 2026-10-20
- MARS steers multimodal safety using only text-derived refusal directions, no multimodal safety data needed — MHUGLab · 2026-10-20
- Lenovo paper: safety benchmark scores mostly measure labeling policy, not detection capability — Vahid Jebraeeli · 2026-10-20