Dev argues transformers can't separate instructions from context, enabling jailbreaks

gerardsans · x · 2026-11-21

Developer gerardsans argues on X that transformers fundamentally cannot distinguish instructions from observations in context, undermining the boundaries a true agent needs — which he calls the main vector for jailbreaks. He adds that transformers needing an external actuator to act is proof they lack any real intelligence. A pointed technical critique in the ongoing agent-safety debate.

Related event: Researcher Argues Transformers Cannot Separate Instructions From Context(2 posts)→

Original post →

More from Models

Models channel →