How to Measure the Effectiveness of Software Factories
lgrammel · x · 2026-07-14
The author discusses how to evaluate a "software factory," suggesting that metrics should be divided into two categories: traditional software delivery metrics and metrics specific to agentic/automated workflows.
Traditional metrics include:
- Throughput: Issues processed and PRs merged per day
- Cycle time: Time from issue creation to closure
For software factories, new metrics should include:
- Agent success rate: Percentage of tasks successfully completed by each agent
- Rework rate: Proportion of work that needs to be redone
- Automation rate: Percentage of automated steps in the workflow
- Wait time: Time tasks spend waiting for human input
The author notes these metrics can be tracked per project or normalized on a per-developer basis.
More from coding & agent
- Tenable and AWS launch a Black Hat build event for open-source security agents and MCP servers — Dave_Maynor · 2026-07-22
- Codex helps build Valdiluce, an open-world game with climbing, gliding and gondolas — Dimillian · 2026-07-22
- HeyGen adds a media-sourcing skill for coding agents with 75k images and 10k tracks — HeyGen · 2026-07-22
- Agent search bottlenecks are now about variance, not raw latency — rohanpaul_ai · 2026-07-22
- LangSmith adds tracing for Pipecat, LiveKit, OpenAI Realtime, and Gemini Live — LangChain · 2026-07-22
- An MCP server signs every AI agent tool call into a verifiable Merkle chain — Funky_Chicken_22 · 2026-07-22