Scrape_01: an open-source self-healing agent scraper that turns URLs into ordered CSVs
Extreme_Leg_6162 · reddit · 2026-10-25
A Reddit user released Scrape01, an open-source self-healing web scraper: give it a URL and a plain-English goal (e.g. "quote, author, tags"), and an agent designs an extractor, verifies it on the live page, then crawls into ordered CSVs.
Two crawl modes:
- Polite mode (default): obeys robots.txt, 1s delay, no JavaScript
- Monster mode: renders in local Chrome/Edge, 150ms delay, harder retries, bigger page budget, ignores robots.txt (skipped hosts logged). For JS-heavy sites and data in embedded JSON. No CAPTCHA bypass; requests stay sequential per host
Around it: CLI (new/run/repair/rebuild/verify/status), local web dashboard (browse/filter CSV, run jobs, live log), Telegram surface, interval loop with auto-repair, OS scheduler install. Extractors cover tables, selectors, JSON-LD, embedded JSON, meta tags, regex, with four pagination modes. 26 tests, zero dependencies.
Limits: JSON configs only, linear crawling only, monster mode needs a local browser.
More from coding & agent
- OpenAI undercuts Anthropic on coding: Replit shifts $100M+ spend, Rippling share falls to 18% — mark_k · 2026-10-25
- Streamrunner ships: AI-powered native macOS reimagining of OBS Studio for $39.99 — jasonkneen · 2026-10-25
- Dev shares crude cross-interface agent setup: Linux box with limited apps plus CUA monitoring — tokenbender · 2026-10-25
- Dev uses agents to monitor group chats and send greetings on his behalf, eyeing voice cloning next — tokenbender · 2026-10-25
- agentic-gate: open-source layer blocks LLM agents' bad tool calls, 150→0 in benchmark — Nice_Current_8581 · 2026-10-25
- There's no moat in code anymore: what's left is distribution, taste, community — paw_lean · 2026-10-25