Claude refuses to bypass robots.txt, sparking worries about AI over-compliance

RexDouglass · x · 2026-10-14

FeryKaszoni shared a screenshot showing Claude flatly refusing to ignore robots.txt: since NY Post and The Mirror explicitly disallow Claude in their robots.txt, the model says it will respect that even when the user asks otherwise.

The author frames this as a slippery slope toward a "100% compliant, AI-controlled world" — predicting Claude may soon refuse SEO link-building work citing Google policies, or even automate enforcement of petty rules. The underlying guardrail behavior in the screenshot is the concrete news hook.

Original post →

More from Safety

Safety channel →