Job-hunting agent fixes reviewed: idempotency keys help, but AUC stays 0.645
nivokvo · reddit · 2026-11-23
A follow-up on an autonomous job-application agent that previously misfired, testing community-suggested fixes: a uuid5-based idempotency key now prevents duplicate sends on retry; fact-checking moved to code that verifies exact quotes from fact files (with a daily canary of 3 known-bad replies, 3/3 rejected on first run); employer-side features added nothing (contact field in 6/147 listings, AUC stuck at 0.645). Key finding: on 147 outcomes and 18 invites, Opus's own 0-10 fit score scored AUC 0.47 (coin flip), a small logistic regression hit 0.645, and the single feature 'fewer than 20 applicants when I applied' alone reached 0.631 — the model's real knowledge is 'apply early'. Code: github.com/nivokvo/job-watch-cloud
More from coding & agent
- Popfold: an open-source three.js engine built with parallel Claude Code sessions — cherichita · 2026-11-23
- Weaviate Podcast #150 dives deep into rerankers and agentic search — CShorten30 · 2026-11-23
- Inference practitioner on GPU co-ops: saturation math, weight caching, and quantization as dark art — sloppenheimer · 2026-11-23
- Siemens and AGIBOT generate robot grippers from text via agentic AI workflow — emmanuelvivier · 2026-11-23
- DeepSeek on 4 DGX Sparks codes great with Opus 5.5 planning, GLM-5.3 not ready — AIFlow_ML · 2026-11-23
- Made a Video for $0.09: Full Workflow with Claude + VidLab API — eptwts · 2026-11-23