From Human-Centric to Agentic Code Review: The Impact of Different Generations of Generative AI Technology on Review Quality
Suzhen Zhong, Shayan Noei, Bram Adams, Ying Zou
cs.SE
2026-07-15
Analyzing 1.02M pull requests across 207 projects, agent-involved reviews cut decision time by 2.5-4.5 days/KLOC but consistently worsen Review Buddies smell, showing AI reviewers trade speed for quality breadth.
As AI-generated code floods development pipelines — Google reports 75% of new code is now AI-generated — human reviewers face an escalating backlog. LLM reviewers and AI agent reviewers have entered the picture, creating a three-generation transition: human-centric, LLM-assisted, and agentic review. Yet large-scale empirical evidence on how this shift affects review efficiency and quality has been absent. This study fills that gap by analyzing pull request history longitudinally across all three eras.
The authors collected 1.02 million reviewed pull requests from 207 GitHub open-source projects with continuous activity from May 2022 to February 2026. Each reviewer account was labeled as human, rule-based bot, traditional ML tool, LLM reviewer, or AI agent reviewer by cross-referencing the GitHub API with official tool documentation. A 384-PR manual validation sample confirmed agentic behavior in 360 of those cases.
AI adoption trajectories were modeled as monthly time series and clustered using soft-DTW clustering, yielding three practices at the optimal silhouette score of 0.40. Within each pull request, review comments were ordered by timestamp to form reviewer-type sequences, then abstracted into 10 collaboration patterns via Markov chain expectation-maximization with BIC-based model selection.
Review quality was measured along two axes: efficiency (days-to-decision normalized by KLOC) and six code review smells including Sleeping Review (>2 days to decision), Review Buddies (same reviewer covers >50% of an author's PRs), and Large Changeset (>500 lines). Logistic regression models with 90/10 train-test splits assessed the relative explanatory weight of collaboration patterns versus traditional factors.
Three adoption practices emerged: Gradual AI Adoption (46% of projects), Rapid LLM Adoption (22%), and Rapid AI Agent Adoption (32%, dominated by large-company projects).
Gradual AI Adoption and Rapid AI Agent Adoption both showed statistically significant efficiency gains in the agent era (-2.5 and -4.5 days/KLOC respectively). Rapid LLM Adoption showed no efficiency gain and a significant increase in review smells of +8.0 percentage points in the LLM era.
At the collaboration pattern level, agent-initiated and multi-agent reviews ranked as most efficient. However, AI-involved patterns carried significantly higher smell prevalence (78%-94%) than human-only review (69%-76%). The primary driver was Review Buddies: from 16% in human-only review up to 60% for LLM-involved patterns and 53% for agent-involved patterns. Once AI reviewers participate, human-AI collaboration patterns become the strongest predictor of both efficiency and Review Buddies, overtaking traditional factors like changeset size and reviewer experience.
This is the first large-scale longitudinal study covering the full three-generation transition in code review. The central finding , that efficiency gains and quality improvements do not co-occur , challenges the assumption that deploying AI reviewers automatically improves the review process. Agent reviewers handle the inspection step, reducing back-and-forth, but this concentrates review responsibility on a small set of AI identities, narrowing feedback diversity. For engineering teams evaluating AI review tooling, the study offers specific guidance: gradual adoption outperforms rapid uniform rollout, and diversifying AI reviewer configurations can mitigate the Review Buddies risk.
The regression models are explanatory, not causal; observed associations may reflect confounding team practices rather than direct effects of AI adoption. The dataset covers only open-source GitHub projects through February 2026, so findings may not generalize to proprietary codebases or to tools released after that date. Code review smells are process-level proxies, not direct measures of defect density. Whether Review Buddies elevation actually increases post-release bugs remains an open question.