He scraped 5.9M jobs with the ChatGPT API to bypass LinkedIn ghost postings
hamed_n · reddit · 2026-09-17
Frustrated by ghost jobs and offshore agencies polluting LinkedIn and Indeed, the author discovered most companies post jobs directly on their own sites — but each career page has a different structure, making large-scale scraping hard.
His trick: dump raw domain names into the ChatGPT API and ask it to return structured job data in JSON (salary, years of experience, etc.). He has now scraped 5.9 million jobs (220k+ remote) and launched a free tool, Hiring.Cafe.
Pro tips shared: multi-select/exclude job titles and functions, filter companies by industry, and distinguish IC vs management roles with desired years of experience.
More from coding & agent
- The 2026 AI engineer roadmap: six stacks from vector search to multi-agent ops — mdancho84 · 2026-09-17
- When the AI Agent Builds the Tool Instead of Doing the Task — Similar_Job_6080 · 2026-09-17
- huggingface_hub 1.32 lets UV scripts declare runtime images for HF Jobs — vanstriendaniel · 2026-09-17
- Open-sourced Qwen-1B-RLCD runs type-safe JSON inference 5x faster on-device — JiliJeanlouis · 2026-09-17
- Microsoft event to demo open-source agent evaluation and governance tools ASSERT and ACS — davemccollough · 2026-09-17
- Cybersecurity Compliance Startup Comp AI Raises $34M Series A Led by Roo Capital and Grand Ventures — JosephJacks_ · 2026-09-17