Goodhart's Law meets AI agents: stop counting PRs, measure the delivery system
rseroter · x · 2026-11-17
Pere Pages published a long-form piece on Goodhart's Law and developer productivity in the age of AI agents: when companies reward PR counts, agents can produce pull requests almost for free, turning output counting from a weak metric into an actively harmful one.
Key recommendations:
- Stop rewarding output counts: lines of code, commits, and PRs per person are now the easiest numbers to inflate without real progress; keep them, if at all, as signals the team reads for itself.
- Measure speed and stability together: how fast changes reach users only means something next to failure and rework rates, measured for the whole team.
- Use short anonymous surveys to find specific delays: don't ask "do you feel faster?" — one study showed developers felt faster while actually being slower.
- Don't accept passing tests as proof: agents sometimes make tests pass by changing the tests; review the tests alongside the code.
- Judge AI by what changed for users, not by how much the team uses it.
Core metaphor: a metric is a thermometer you read to learn about the room; the moment you pay bonuses on the thermometer, it stops showing the real temperature. Measure the delivery system and outcomes, but don't obsess over any single number.
More from coding & agent
- Dev calls for an 'omni-feed' product: one filterable pane for all agent actions — DavidWells · 2026-11-17
- LangChain adds host allowlists to Managed Deep Agents, blocking injected exfil via proxy — hwchase17 · 2026-11-17
- Running 10 AI agents in parallel: "I'm the bottleneck of my own company" — stewones · 2026-11-17
- Bitter lesson comes for multi-agent systems: RL-trained swarms beat hand-designed orchestration — emax · 2026-11-17
- Glass Health launches developer app with Agent Builder, Context Libraries and Glass 6.0 API — GlassHealthHQ · 2026-11-17
- Official Xcode plugin lands in Claude Code plugin marketplace: build, test, SwiftUI previews — ClaudeDevs · 2026-11-17