Cloudflare: bot traffic to surpass humans by May 2026, will pay sites agents crawl
Cloudflare Blog · rss · 2026-09-28
Cloudflare's 16th annual Founders' Letter argues AI is reshaping the internet:
- Web explosion: after plateauing from 2012–2025, new sites surged in mid-2025 — driven less by AI slop than by vibe-coding tools enabling non-coders to build; over 7 million developers now build on Cloudflare's developer platform.
- Bots overtake humans sooner: automated traffic was forecast to pass human traffic in H2 2027, but agent and AI-crawler growth pulled that forward to May 2026; at current trends, bot traffic will be 1,000x human traffic within five years.
- Tragedy of the commons: an agent recommending one lunch spot may crawl 1,000 restaurant menus, leaving 999 sites serving load with no payoff; over half of what good bots fetch is unchanged content.
- Risk to newcomers: agents favor whoever has the most information (usually incumbents), threatening consolidation. Cloudflare says it wants '500,000 AI companies, not five,' and is shipping crawler efficiency improvements plus a way for sites to get paid when agents use their content.
More from Companies & People
- Ages 36-44 are a career uncanny valley, says investor; build and sell to break through — viksit · 2026-09-28
- David Sacks: A Billion Meta Muse Users Could Fix AI's PR Problem — kevinnbass · 2026-09-28
- Investor: Musk keeps winning the few irreversible decisions that actually matter — gabriel1 · 2026-09-28
- Investor: Silicon Valley's consumer AI pivot opens a window for coding agent startups — edgarpavlovsky · 2026-09-28
- Hono creator Yusuke joins VoidZero, stays Cloudflare developer advocate — cnakazawa · 2026-09-28
- Yacine's Management Idea: Reorg the Losing Team Into the Winning One — yacineMTB · 2026-09-28