20 Open-Source Web Scraping and Browser Agent Tools Worth Knowing in 2026
MaryamMiradi · x · 2026-11-13
The author maps the open-source ecosystem for building AI agents that search, extract, fill forms, log in, or complete browser tasks, organizing 20 tools into five categories:
- Browser AI agents: Browser Use, Skyvern, Stagehand, Browser Use Web UI — when agents need to navigate and act
- Scraping + extraction: Firecrawl, Crawl4AI, Scrapy, Trafilatura — for clean web data over click-through workflows
- Browser control: Playwright, Selenium, Crawlee, nodriver — deterministic control around the agent
- Browser infra + sessions: Steel Browser, Browserless, Camoufox (anti-detect Firefox), Selenium-Driverless — turning local automation into a scalable service
- Testing + benchmarks: BrowserGym, WebArena, Stagehand Agent, Browserbase Python SDK — verifying what the agent actually did
Real browser work means handling changing pages, failed clicks, expired sessions, logins, and retries; the author may benchmark Browser Use vs Skyvern vs Stagehand vs Crawl4AI vs Firecrawl vs Playwright next.
More from coding & agent
- Arena's Interaction-as-a-Judge: evaluating computer-use agents by how they play — arena · 2026-11-13
- Current launches as an opinionated tool to run hundreds of AI agents in control — edgarpavlovsky · 2026-11-13
- HermesTube plugin brings YouTube subscriptions into Hermes agent, skipping the algorithm — Teknium · 2026-11-13
- RepoMem: Giving Coding Agents Long-Term Memory from Git History to Improve Bug Localization — burkov · 2026-11-13
- The Zvi: Opus 5.5 keeps finishing my coding ambitions before I can use Fable 5.2 — TheZvi · 2026-11-13
- Mnemos gives Claude Code, Codex and Hermes one shared cross-agent memory — RileyRalmuto · 2026-11-13