He scraped 5.9M jobs with the ChatGPT API to bypass LinkedIn ghost postings
hamed_n · reddit · 2026-09-17
Frustrated by ghost jobs and offshore agencies polluting LinkedIn and Indeed, the author discovered most companies post jobs directly on their own sites — but each career page has a different structure, making large-scale scraping hard.
His trick: dump raw domain names into the ChatGPT API and ask it to return structured job data in JSON (salary, years of experience, etc.). He has now scraped 5.9 million jobs (220k+ remote) and launched a free tool, Hiring.Cafe.
Pro tips shared: multi-select/exclude job titles and functions, filter companies by industry, and distinguish IC vs management roles with desired years of experience.
More from coding & agent
- General Reasoning explains how to train agent swarms with RL and latency pressure — rosstaylor90 · 2026-11-02
- "Coding isn't the bottleneck anymore — usage windows are," says founder Kyle Gawley — kylegawley · 2026-11-02
- Demo passed ≠ production-ready: 5 questions to answer before letting agents run your business — _kellhus · 2026-11-02
- An AI agent passed 100 test chats—still not ready to run the business — djgoel123 · 2026-11-02
- Agent given total creative freedom produces a 1:44 code-drawn animated short film — ivegotnoidea1 · 2026-11-02
- Meme: Wake Up, Use Claude Code, Repeat All Day — tekbog · 2026-11-02