Why Copilot Code Review Worsened After a Tool Swap
GitHub Blog AI/ML · rss · 2026-07-10
GitHub shared a counterintuitive case: replacing Copilot code review with a more general, better-maintained tool actually made performance worse.
What Happened
- The original code review tools were replaced by shared grep / glob / view tools from Copilot CLI.
- In offline benchmarks, review costs increased, and fewer valid issues were caught.
- Tracing logs revealed the issue wasn't the tools themselves, but incorrect tool descriptions for the review context. The model searched aimlessly like it was "browsing a repository," accumulating too much irrelevant context.
The Fix
GitHub revamped the prompts and tool usage guidelines to fit a "reviewer workflow":
- Start from the diff and form specific review questions first;
- Use glob when paths are uncertain, and grep to search for candidate files/symbols;
- Perform batch, cheap discovery first, then use view to read precise evidence;
- Minimize the back-and-forth switching between searching and reading.
Results
After the adjustment, average review costs dropped by about 20% while maintaining the same review quality.
The key takeaway here isn't "stronger tools are better," but rather: the same set of tools requires entirely different behavioral constraints across different tasks.
More from coding & agent
- A roundup of AI agents and MCP resources, including how to evaluate agents — _jaydeepkarale · 2026-07-21
- A full course shows how to build and deploy an AI agent with OpenAI and LangChain — _jaydeepkarale · 2026-07-21
- A beginner guide to AI agents points readers to a Stanford webinar — _jaydeepkarale · 2026-07-21
- A practical guide on how to evaluate AI agents — _jaydeepkarale · 2026-07-21
- MCP is headed toward easier scale, event-driven extensions, and workable file uploads — EricBuess · 2026-07-21
- Developers debate the missing composition model for AI agents — threepointone · 2026-07-21