False greens: what do you actually verify when a coding agent says tests passed?
Weary-Information-15 · reddit · 2026-11-03
The author, building an independent verification prototype for coding agents, reproduced two constructed stress cases: pytest reporting failures yet exiting 0, and a config change silently deselecting failing tests for a green result. They ask practitioners what they verify before accepting agent work, whether false greens occur in real workflows, who validates legitimate test modifications, and what manual verification remains when CI already exists.
More from coding & agent
- OpenAI's new Git backend Pando cuts push latency ~80%, serves ~1PB of Git data daily — OpenAIDevs · 2026-11-03
- OpenAI engineer details Pando, the source-control rewrite built for the agent era — OpenAIDevs · 2026-11-03
- Trajectory launches post-training platform with GS1 and Continual Graders research previews — simonguozirui · 2026-11-03
- SMBs are rebuilding their entire GTM stack with agent-native tools, says new thread — garrytan · 2026-11-03
- Compacting Claude Code at 400K instead of 967K cut this dev's cost per request by 43% — Awkward-Let-4628 · 2026-11-03
- Capy 0.7.0 ships Captains orchestration GA; YC's Garry Tan reports 5X issue-fixing throughput — garrytan · 2026-11-03