When two agents agree the job is done but the customer still gets the wrong product
krishnan · x · 2026-11-07
Drawing on Anthropic's harness-design report, the author examines a subtle failure mode in multi-agent development workflows.
- The report's approach: before each coding sprint, a generator and an evaluator negotiate a contract defining what will be built and how success is checked; the evaluator then uses Playwright to interact with the running application.
- The author likes the sequence — agree on observable behavior before writing code, then test the actual application rather than the developer's description.
- But there's a trap in who writes the contract. Example: a customer asks to export all records; the builder proposes a download button; the evaluator confirms it produces a valid file. Both agents agree and all tests pass, yet the export silently excludes archived records.
- Takeaway: two agents agreeing the job is done can still leave the customer with the wrong product — contract quality and coverage are what matter.
More from coding & agent
- Vercel Tracing goes GA on all plans, enabled by default in Next.js 16.5 — cramforce · 2026-11-07
- Decompiling Battlefield 1942 with Claude Code costs ~$100 in API tokens — alvelda · 2026-11-07
- Fine-tuned Qwen3.5-4B router model trained on 17.6k rows for about $5.50 on one A100 — MaziyarPanahi · 2026-11-07
- Access your coding agent's localhost via Tailscale instead of waiting for Vercel deploys — chongdashu · 2026-11-07
- The Agentic Web: personal AI assistants that understand what every website can do — JnBrymn · 2026-11-07
- GPT-Image-2.6 lands on Netlify AI Gateway; demo turns any birth city into a looping poster — thisiskp_ · 2026-11-07