Swap paragraphs for words and Qwen denies injection — yet still guesses the word better

Sauers_ · x · 2026-11-19

Boundary-condition findings from Sauers' thread: when the injection target switches from paragraphs to single words, Qwen no longer wants to report that an injection occurred — yet still gets better at guessing the injected word.

Paragraphs work better than words, possibly because paragraph boundaries act as natural breakpoints; instructing focus before the paragraphs weakens the effect. Notably, attention lands on the word itself, not the label, while the prefill requires answering about the label — so it's not simply "mindlessly raising the logprobs of high-attention words."

Related event: Experiments suggest Qwen3-32B has privileged introspective access to its own attention(10 posts)→

Original post →

More from Research

Research channel →