Claude Code taps Oxylabs via MCP to scrape bot-protected sites into clean JSON
mhdfaran · x · 2026-10-29
- Claude Code can now pull clean data from sites it couldn't open before by running the Oxylabs Web API through MCP, returning structured JSON or Markdown from JS-heavy and bot-protected pages.
- Demo: asking for "iPhone 18 Pro retailers in Germany" makes it fetch stores and carriers with listing URLs.
- It can also turn Apple's product page into a clean price comparison table on its own.
- The original post includes a full workflow with prompts — a reproducible recipe for agent-based web scraping.
More from coding & agent
- PcGöz MCP server gives Claude text-based Windows desktop vision, ~30x fewer tokens than screenshots — emperorbug46 · 2026-10-29
- Code Talks talk: shipping offline in-browser AI with Transformers.js and WebGPU — nicodotdev · 2026-10-29
- AgentWeb launches an action layer that lets AI agents skip browser emulation entirely — rohanpaul_ai · 2026-10-29
- Developer's agent babysits Codex Cloud threads overnight while he sleeps — gabrielchua · 2026-10-29
- Fortune 100 firms still shun coding agents — but engineers have likely smuggled them in already — chrisalbon · 2026-10-29
- Vercel's open-source coding agent harness fx is now a built-in provider in T3 Code — moeinteractive · 2026-10-29