Cloudflare to Block AI Crawlers by Default, Requiring Payment for Content Access
emmanuelvivier · x · 2026-07-06
Cloudflare announced that starting this September, it will block AI crawlers from scraping website content by default. AI companies will now have to pay publishers to access their content. This move is seen as a major turning point for the internet content economy, potentially fundamentally altering how AI models source training data and reshaping the commercial negotiations between AI firms and content creators.
More from Safety
- Fangcun AI, China's first independent frontier AI safety lab, launches with DeepSeek eval report — JacquesThibs · 2026-10-26
- ESR on the Bugpocalypse: locking down frontier models only hands the game to Chinese open weights — colinmcnamara · 2026-10-26
- OpenCode breach exposed AI API keys of 1.4% of Console users — thione · 2026-10-26
- DHS orders DeepSeek products removed from US civilian agency systems — thione · 2026-10-26
- One hacker breached 100+ company websites using AI, scope unknown — thione · 2026-10-26
- METR red-teams OpenAI's agent monitoring, finds bypasses and gaps — thione · 2026-10-26