10 Open-Source MCP Projects to Give AI Agents Web Search and Crawling
Shruti_0810 · x · 2026-10-31
A curated list of 10 open-source GitHub projects that give AI agents live web access, mostly as MCP servers:
- Scout MCP: adds web search, crawling, and data extraction to coding agents
- Exa MCP Server: connects assistants to Exa's search and crawling capabilities
- MCP Web Search: combines multiple search providers with webpage content extraction
- Searchlight: multi-engine search with no API keys required
- MCP Retrieval: a Go-based, locally running server for web search, image search, and page-to-Markdown scraping (no API keys needed)
- Web Research MCP: gives Claude Google search, screenshots, and more
A resource roundup for developers who want to equip their agents with real-time internet access via ready-to-plug MCP servers.
More from coding & agent
- Pass/Fail Beats 1-5 Ratings for Agent Regression Tests, Argues Practitioner — intellectronica · 2026-10-31
- Cursor confirms both extension versions 2.1.312 and 2.1.313 contained malicious payload — cyb3rops · 2026-10-31
- Dev ditches the code pane: running two parallel Claude Code sessions beats editor + assistant — 4310sy · 2026-10-31
- Claude Opus drives Blender MCP to build 3D models straight from 3DGS — janusch_patas · 2026-10-31
- Hot-swapping 18GB models RAM↔VRAM on one 24GB GPU: 1.5s physics vs 29s llama.cpp reality — bjivanovich · 2026-10-31
- Team wires MCP-connected AI agents to auto-reject ~98% of job applications — AnuranBuilds · 2026-10-31