After exhausting the open web, AI scrapers now hammer €7/month self-hosted servers
cen6wkf · reddit · 2026-08-18
David Gerard (Pivot to AI) describes how his site, running on a €7/month server, is being hammered non-stop by AI scrapers wearing a fake Chrome mask, hopping IPs so he can't block them, and ignoring robots.txt entirely.
The poster draws an analogy to a 2005 plagiarism incident: having eaten the free web, scrapers are now consuming the cheap self-hosted tail. The core point: exposure runs downhill — platforms with lawyers can defend themselves, small servers with none quietly get taken from at scale. The thread collects similar stories from self-hosters.
More from AGI Musings
- OpenAI Foundation holds 26% of OpenAI; AI is about to unlock hundreds of billions in philanthropy — iskander · 2026-08-18
- Pixels, compute, and the mechanism of parallel universes — yunta_tsai · 2026-08-18
- Prediction: AI to Discover New Science Daily, Enable Zero-Employee Firms Next Year — rand_longevity · 2026-08-18
- Opinion: Travel booking is already automated; AI hype ignores reality — binarybits · 2026-08-18
- LLMs are turning ancient philosophy questions into testable experiments — Cyborgized · 2026-08-18
- METR proposes apple-picking model for AI R&D diminishing returns — soumitrashukla9 · 2026-08-18