GPTBot Hit This Site 3.1M Times in 24 Hours; a robots.txt Rule Stopped the Crashes
tsi_org · x · 2026-10-03
A site owner reports OpenAI crawlers sent over 3.18 million requests in 24 hours, crashing his servers every 10 minutes. Logs show GPTBot/1.0 hit 3,123,466 times and GPTBot/1.2 58,746 times, plus a handful of ChatGPT-User and OAI-SearchBot hits.
The fix was simple: adding rules to robots.txt to block those bots stopped the fake traffic entirely. His advice to site operators: edit robots.txt to protect your servers — not all traffic is good traffic.
More from Models
- Grok Bot usage limits reset, users celebrate a free weekend of Grokking — mark_k · 2026-10-03
- Every's vibe check: GPT-6 Astra impresses at writing but trails Anthropic's Fable on long-running tasks — every · 2026-10-03
- GPT 6.1 Sol is 2.5x slower and pricier than GPT 6 Sol in BridgeBench tests, contradicting OpenAI's efficiency claims — RexDouglass · 2026-10-03
- OpenAI launches GPT-6.1 Sol at $2/$10 per million tokens, beating Astra pricing and key benchmarks — dl_weekly · 2026-10-03
- KernelBench-Verified: no frontier model beats PyTorch when evals get strict, Meta/Stanford find — lmoroney · 2026-10-03
- User Finds ChatGPT Beats Claude at Writing Natural-Sounding Cover Letters — Whole_Intention_7949 · 2026-10-03