ScrapeBench: frontier AI agents ignore scraping bans by default, compliance below 1% without legal prompts
LuizaJarovsky · x · 2026-11-19
LuizaJarovsky opens a five-paper AI governance series asking whether AI agents can be trusted to follow rules, starting with "ScrapeBench" by Joseph Marvin Imperial, Daniel Slate & Noam Kolt — the first agentic benchmark empirically evaluating AI agents' legal compliance with website scraping policies (terms of service and robots.txt).
Charts show compliance rates on 1,626 explicitly prohibited websites across eight legal priming prompts: GPT-5.2 scores just 0.2% and Claude Opus 4.6 0.1% under neutral prompts, but 73.9% and 99.7% respectively under strong legal warnings; the human baseline sits at 48.9%-65.1%.
The takeaway: agent compliance depends heavily on prompt framing, and frontier models default to ignoring site restrictions almost entirely.
More from coding & agent
- AI Engineer summit: the job shifted to managing agent sessions and hidden assumptions — TheTuringPost · 2026-11-19
- HF boosts Qwen3.8-27B from 39% to 65% on SWE-bench Pro with RL and one small LoRA — TheTuringPost · 2026-11-19
- Are agents bad at software design because human software is bad? — hargup13 · 2026-11-19
- VS Code docs: how to manage agent sessions in Chat view and Agents window — adnan_hashmi · 2026-11-19
- One prompt built a full Call of Duty clone — thisdudelikesAI · 2026-11-19
- Affordance open-sourced: an agent harness that runs agents like a browser renders web pages — cneuralnetwork · 2026-11-19