Long-running agents need trustworthy verification, not just more loops
hugobowne · x · 2026-07-25
Wes McKinney argues that long-running agent systems should be judged by verification, not by whether they keep looping.
- “Keep going” is not enough: self-grading can push a system into the “vibe zone,” where it produces more output and more confidence without getting closer to a correct result.
- What good long-running systems need:
- tests and constraints for tasks like “Ralph Wiggum” loops
- a held-out metric for autoresearch
- an objective function for hill-climbing workflows
- Core claim: if verification is not trustworthy enough, human verification still has to stay in the loop.
- Broader point: renaming loops as graphs does not solve the problem; graphs help with branching, dependencies, parallelism, and state transitions, but they still depend on reliable evaluation.
The post also promotes a free Lightning Lesson next Wednesday on “Loop Engineering, Without the BS,” with registration for live attendance or the recording afterward.
Related event: Validation is Key for Long-Running AI Agents(2 posts)→
More from coding & agent
- How to ace the "send us something cool" AI interview: research interviewers with agents, then focus — brandon_galang · 2026-10-27
- OpenAI DevDay Berlin: Dots agents, Codex upgrades, custom plugins & Sites — No_Ranger_4217 · 2026-10-27
- Give AI a clean personal-data-free machine with full control: agents running 24/7 teach you more than any course — EXM7777 · 2026-10-27
- 20 papers across 6 areas to master production-grade agentic AI engineering — MaryamMiradi · 2026-10-27
- LangChain SF meetup to compare decision model Jev vs LLM judges for agent evals — LangChain · 2026-10-27
- How Clang recognizes your popcount loop and swaps it for a single CPU instruction — jimmykoppel · 2026-10-27