7 steps to make coding agents trustworthy: 11,000 PRs/month at 0.5% revert rate
TheTuringPost · x · 2026-11-12
Turing Post distills an engineering lead's (@poteto) full methodology for making coding agents trustworthy — the core idea: make good code the easiest thing for an agent to produce before scaling.
- Clean the codebase agents learn from — it's their biggest input; agents copy established patterns including workarounds and deprecated code.
- Turn corrections into enforceable checks — lint rules, type constraints, tests, better APIs; a growing AGENTS.md still relies on the agent obeying every instruction.
- Remove predictable mistake opportunities — her team restricts React's useEffect to framework internals (agents repeatedly misuse it) and enforces import boundaries between processes, killing whole categories of errors.
- Give agents reliable ways to run and inspect the product — debugging tools, reusable CLIs, and a user-perspective "feature map" so "that thing in the left sidebar" has shared meaning.
- Review evidence of behavior — screenshots, videos, state diagrams; a convincing explanation of a fix is easy to generate, seeing it work is far more useful.
- Spend serious effort on verification — runtime verification, property-based UI testing, TLA+ model checking; most of her token usage goes to verification.
- Automate once the loop is reliable — agents now exercise releases, find bugs, open and merge fixes automatically; building that trust took 6 months.
Reported results: 11,000 PRs shipped in a month, 0.5% revert rate. On "software factory" fantasies: if your loop produces slop, automating it produces more slop, faster, with a larger token bill.
More from coding & agent
- Together Compiler: agent-driven continual compilation for AI inference — togethercompute · 2026-11-12
- Kalshi partners with AnyAlpha to let AI agents trade prediction markets 24/7 — ycombinator · 2026-11-12
- Google solicits Kubernetes users to help shape Agent Substrate for Managed Agents — rakyll · 2026-11-12
- Building apps in the agent era: elegant abstractions still beat raw code generation — cnakazawa · 2026-11-12
- Grok agent autonomously browses, orders and pays for a book on an unfamiliar site — xkonjin · 2026-11-12
- Stop Managing Sandbox Fleets for Cloud Agents, Argues gakonst: Go Harness-First — andreisavu · 2026-11-12