Sentry CEO: my well-behaved web scraper gets blocked by default across the web
zeeg · x · 2026-09-08
Sentry CEO zeeg notes an industry irony: his very well-behaved web scraper is blocked by default on many sites, while "no one plays good citizen" anymore.
The post highlights how sites increasingly block all crawler/agent traffic indiscriminately — good-faith scraping etiquette no longer pays off, a practical concern for developers building data collection and agent workflows.
Related event: Sentry CEO Complains Well-Behaved AI Crawler Still Gets Blocked(3 posts)→
More from coding & agent
- Netlify adds GPT-6 Astra, Gemini 3.8 Flash, Claude Fable 5.1 to AI Gateway, fixes Agent Runner credits — thisiskp_ · 2026-09-09
- PSAISuite: swap LLMs in PowerShell by changing one -Model string — dfinke · 2026-09-09
- MoonPay's Paybox AI payment vault goes live in Claude, ChatGPT, Grok — kleffew94 · 2026-09-09
- 20-year IT veteran ships Train Tycoon game on Steam with Claude's help — maartendeblock · 2026-09-09
- GPT-6 'Astra' builds an interactive human anatomy app with ~4,000 structures — omarsar0 · 2026-09-08
- Tool adds Gmail, Drive, Docs and Sheets connections; email lands in drafts for now — tekbog · 2026-09-08