GitHub Copilot’s code review upgrade: why better tools aren’t always the answer
When GitHub Copilot’s code review agent got an upgrade—swapping its custom tools for the more robust set behind Copilot CLI—engineers expected sharper reviews and lower costs. Instead, they measured higher costs and fewer issues caught. The culprit wasn’t the tools themselves, but the way they were being used. After rewriting the agent’s instructions to match how human reviewers actually read pull requests, the regression turned into a win: roughly 20% lower average review cost, with no drop in quality.
A familiar trap: shiny tools, broken workflows
It’s tempting to assume that better tools automatically yield better results. In practice, tools tuned for one workflow can backfire when applied to another. Copilot code review originally relied on custom agents inspired by earlier models like SWE-agent and Copilot Autofix, which were designed for sparse, high-impact tool calls. Those tools aggressively bundled context into each call—extra lines, directory listings, search hits—because earlier agents rarely fetched context on their own. When GitHub migrated to the shared CLI tools—grep, glob, and view—the extra context became pure overhead, bloating token counts without improving relevance.
The browsing loop that cost more than it caught
Internal benchmarks showed the agent falling into a browsing loop: it kept re-scanning the same files, pulling overlapping context, and producing fewer meaningful comments. The shared CLI tools were leaner and more maintainable, but their design assumed agents would chain calls efficiently. Copilot code review, however, needed a different rhythm—one closer to how humans review diffs: focused, incremental, and driven by the structure of the change itself. By rewriting the instructions to prioritize targeted exploration and suppress unnecessary context, the team flipped the regression into a measurable improvement.
## Why it matters
This isn’t just a story about GitHub Copilot—it’s a reminder that tool upgrades can degrade performance if workflows aren’t realigned. The stakes are clear: every wasted token, every redundant scan, adds up across thousands of reviews. For engineering teams betting on AI-assisted code review, the lesson is practical: don’t assume better infrastructure solves the problem; audit the instructions first. And for toolmakers, the episode underscores the need to design agents that adapt to human-like reviewing patterns, not the other way around.
Source: GitHub Blog. AI-assisted editorial synthesis — TechnoExpress.