ScrapeGraphAI: open-source library that scrapes any site with one plain-English prompt
thisdudelikesAI · x · 2026-09-18
ScrapeGraphAI (29.5K stars, MIT license) uses LLMs plus graph logic to build the entire scraping pipeline — no CSS selectors, no XPath, no breakage when layouts change.
- Write one prompt (e.g. "extract company description, founders, social links"), point it at a URL, get clean structured JSON.
- Works on websites and local files (HTML/XML/JSON/Markdown); SmartScraper for single pages, SearchGraph for top search results, ScriptCreator to generate reusable Python scripts.
- Bring your own LLM: OpenAI, Groq, Gemini, Azure, or fully local via Ollama; integrates with LangChain, LlamaIndex, CrewAI, n8n, Zapier.
Related event: ScrapeGraphAI Hits 30K Stars: One Prompt Replaces Entire Scraper Code(2 posts)→
More from coding & agent
- LoRA on abliterated Qwen 27B recalls private codebase rules for ~7.5h training — Similar_Job_6080 · 2026-09-18
- Open-source repo collects the best JEV use cases in one place, PRs welcome — matchaman11 · 2026-09-18
- Losing 3 days to a dead coding agent session sparks push for 'AgentGit' workflow forking — LeviYagami · 2026-09-18
- User: Astra in Codex is unusable even on a 20x subscription — OpenAI should copy Anthropic's usage-quota approach — CtrlAltDwayne · 2026-09-18
- Controlling the iOS simulator with Jev + AXe: ultra fast at a fraction of LLM cost — stefanjblos · 2026-09-18
- Jev reviews PRs at $0.00007 each, ~200x cheaper than Claude Opus with 14 typed checks — stefanjblos · 2026-09-18