Broad AI crawler bans would gut the scraping tools newsrooms rely on
BlancheMinerva · x · 2026-10-17
EleutherAI's Stella Biderman amplifies Re:Create Coalition's critique of the Stealth Bot Prohibition Act: Condé Nast, Hearst, Gannett and The Seattle Times are lobbying Congress to restrict automated crawling, yet journalists have used web scraping for 20+ years to expose corruption and lobbyist influence, and modern newsrooms increasingly depend on AI tools trained on web data. Broad crawler bans risk cutting off those exact tools.
Related event: AI crawlers bans spark backlash over hasty regulation harms(2 posts)→
More from Safety
- METR: AI labs redacted key claims before approving frontier risk report — kevinnbass · 2026-10-17
- Macroscopic Ventures: Anthropic backer with stakes in Apollo Research, Gray Swan and Redwood — kevinnbass · 2026-10-17
- Evaluator payment policies compared: METR takes no lab money, Apollo charges market rates — kevinnbass · 2026-10-17
- Thread proposes SEC-style auditor independence rules for AI safety evaluators — kevinnbass · 2026-10-17
- Government AI testers share personnel ties with Anthropic trust and OpenAI board — kevinnbass · 2026-10-17
- OpenAI safety committee chair Zico Kolter co-founded evaluator Gray Swan AI — kevinnbass · 2026-10-17