Long-running agents need trustworthy verification, not just more loops
hugobowne · x · 2026-07-25
Wes McKinney argues that long-running agent systems should be judged by verification, not by whether they keep looping.
- “Keep going” is not enough: self-grading can push a system into the “vibe zone,” where it produces more output and more confidence without getting closer to a correct result.
- What good long-running systems need:
- tests and constraints for tasks like “Ralph Wiggum” loops
- a held-out metric for autoresearch
- an objective function for hill-climbing workflows
- Core claim: if verification is not trustworthy enough, human verification still has to stay in the loop.
- Broader point: renaming loops as graphs does not solve the problem; graphs help with branching, dependencies, parallelism, and state transitions, but they still depend on reliable evaluation.
The post also promotes a free Lightning Lesson next Wednesday on “Loop Engineering, Without the BS,” with registration for live attendance or the recording afterward.
Related event: Validation is Key for Long-Running AI Agents(2 posts)→
More from coding & agent
- Inspired by OpenAI's 10,000-agent run, dev open-sources a crowdsourced agent problem-solving platform — Benjaminsen · 2026-09-11
- Lucid: open-source Mac app keeps your laptop awake only while AI agents run — Pitiful_Hedgehog_600 · 2026-09-11
- Alex Townsend posts 200 open problems in numerical linear algebra for humans and AI agents — IgorCarron · 2026-09-11
- Kimi K2.8 Preview rolls out: near-K3 coding performance, 1M context for all tiers — teortaxesTex · 2026-09-11
- Looking for a classifier of software engineering task shapes to pick models per task — StewartalsopIII · 2026-09-11
- Steal this idea: prompt-to-hardware where agents assemble custom devices — paraschopra · 2026-09-11