Prove a coding agent actually ran the tests: check artifacts, not approvals
ComprehensiveShake76 · reddit · 2026-10-16
Running approval-free Codex workers on a Mac, the author moved the "is it done?" check from per-command approvals to artifacts:
- Done is an artifact, not a sentence: a task is done when you can see the real test run — e.g., actual output like "6 tests, OK, exit 0, 55s" from the worker's own Linux container.
- Research gets the same treatment: a file with size, source count and hash, noting sources are listed, not fact-checked.
- Nothing lands on the Mac until reviewed; files stay in the agent's workspace.
- Approvals were the wrong check — they never indicated quality, just fatigue. End-of-task checks work better, and feel safe only because the container holds none of the author's files (a strong boundary, not a guarantee).
Unsolved: an agent can write weak tests that pass. How do you check the tests themselves? The author asks the community.
More from coding & agent
- Why company AI agents go stale: adoption needs in-house knowledge, not just better agents — EdenEmarco177 · 2026-10-16
- Cheng Lou open-sources Handwired: hand-written weights turn code into training-free neural nets — dotey · 2026-10-16
- Building a portable context vault in Notion so every AI agent knows you from day one — ivanhzhao · 2026-10-16
- Lovable apps can now talk to customers on WhatsApp with inbox, reminders and templates — AlexandrePesant · 2026-10-16
- FDE-Gym benchmark: best agent fully solves only 14 of 95 real customer-delivery tasks — dair_ai · 2026-10-16
- Deep Research API showdown: OpenAI, Perplexity, Parallel, Tavily, Gemini, Valyu compared — bar_raiser333 · 2026-10-16