Jevcrawl: a web crawler for AI agents with no LLM in the crawl loop
rayenmlayeh · hn · 2026-10-14
Developer RayenMlayeh launched Jevcrawl on Show HN: a web crawler built for AI agents whose crawl loop runs entirely without LLM calls.
- The crawler handles fetching and link selection with traditional heuristics, leaving LLM calls to the downstream agent, cutting the cost and latency of LLM-in-the-loop scraping (a nod to Jevons-paradox-style compute economics).
- Open source on GitHub; a lightweight alternative to tools like crawl4ai for the data-collection layer of agent pipelines.
More from coding & agent
- Decisions API hits GA as agents automate cloud video editing via feedback loops — dkundel · 2026-10-23
- Swarms API changelog: 8 months of streaming, multi-agent orchestration, and dynamic model pricing — KyeGomezB · 2026-10-23
- A curated library of 50+ free AI agent resources: videos, repos, one Google Doc — ifioknkem · 2026-10-23
- Vercel adds no-account deploys: drag and drop a file to get a live URL — steren · 2026-10-23
- Dev runs 5 Claude Code sessions that coordinate via MCP Agent Mail to root his network cameras — doodlestein · 2026-10-23
- Vercel ships 'vercel deploy --temporary' for ephemeral agent-built deployments — cramforce · 2026-10-23