Characterizing the Quality Profile of AI-Generated C++ in Production
Michael Tran, Fred Lewis, Kun Yang, Saksham Thakur, Aditya Kini, Aditya Patil, Milad Hashemi, Parthasarathy Ranganathan
cs.SE, cs.AI
2026-08-07
Across 3.52M changes in a year, AI-generated C++ leans on explicit loops over standard APIs, adding 5-8% compute cost; taxonomy feedback cuts targeted static warnings 11.1%.
AI coding assistants make engineers faster; almost nobody disputes that. The dispute is on the other end: how good is the code these assistants produce? Prior answers came from controlled settings, fixed prompts, benchmarks, or snippets pulled out and judged in isolation. Those settings miss the full production lifecycle where code is edited, reviewed, merged, and finally runs online. Production is where code actually incurs cost, but it has an observability wall: it is hard to know who (human or model) wrote each line, harder still to tie that line to later compute cost.
This paper tears that wall down inside one company. The team (Milad Hashemi, Parthasarathy Ranganathan, and others; from the author list and the "billions of daily users" framing, this is almost certainly Google) ran a large-scale empirical study across a review-gated monorepo on a full year of C++ changes, April 2025 to April 2026: 3.52 million changes covering 10.46 million lines of C++ with authorship provenance. The core question is whether AI-generated code has a recognizable problem profile that survives review and cashes out as production cost.
The keystone is provenance. Rather than guessing after the fact which lines are AI-written, the authors record byte-level authorship as engineers work, then project it onto the landed revision. Sources span four interaction modes, inline completion, conversational generation, agentic editing, and transformation-based editing; an "AI share" for each function is the fraction of informative bytes attributed to AI features. This is stronger than post-hoc source-origin heuristics, though the authors concede that human and AI features overlap on the same bytes, so some ambiguity remains.
Around this provenance signal they build a three-level static-analysis taxonomy: five top-level quality attributes (efficiency, correctness and safety, maintainability, API modernity, policy and portability), 15 middle-tier issue categories (for example Copy & Allocation Overhead, Interface & Coupling Burden), and concrete static-check patterns underneath. An AST-level source-efficiency pass counts loops, standard-library calls, move/copy behavior, container insertion, and map access. For compute, they longitudinally track each function cohort's share of application CPU and heap memory in production.
AI code share grew over the year from 29% (April 2025) to 69% (March 2026); for C++ alone, 28.6% to 62.8%. The scale says this is not a fringe phenomenon.
Structurally, AI changes are larger: median lines changed 89 vs 33, files touched 3 vs 2, new-code ratio 0.83 vs 0.60; individual functions are shorter, though (11 vs 15 lines). For static issues, the AI/human rate ratio (>1 means AI is higher):
| Dimension | AI/Human |
| Efficiency & resources | 1.23 |
| incl. Copy & Allocation | 1.39 |
| incl. I/O & formatting | 3.16 |
| Interface & coupling | 1.15 |
| Deprecated API use | 1.41 |
| Correctness & safety | 0.94 |
One contrast matters: on correctness and safety AI is slightly lower (0.94). It is not bad everywhere. The systematic weakness concentrates in efficiency, and the two categories Interface & Coupling plus Copy & Allocation account for 82% of the positive absolute rate gap.
The source-level view is blunt: AI uses explicit loops about 2.0x as often, standard-library calls about 0.4x, and inefficient container insertion and map access also about 2.0x. In plain terms, AI prefers hand-rolled for-loops over std algorithms.
On review and reliability, AI changes draw 1.92x as many blocking comments, 1.39x total comments, 1.24x reviewer iterations, and 1.19x time-to-merge; sanitizer hits and build-failure rates are each about 1.3x. Revert rate is slightly lower for AI (0.9x).
Compute is the expensive part. By early 2026, AI-heavy functions reached 1.31x baseline CPU cost versus 1.25x for human functions, a relative excess of about 5%; memory was 1.36x vs 1.25x, about 8% relative excess. At a company with billions of daily users, 5% CPU is real money.
The verdict here is specific, not a blanket "AI code is good or bad." It localizes the weakness to the efficiency category and to one actionable habit: explicit loops instead of standard-library calls. For practitioners this implies two things. First, code review does not catch these localized inefficiencies; the authors checked review depth against inefficient-code survival and found no clear correlation, so human reviewers miss them and upstream automated intervention is needed. Second, the problem is fixable by feeding it back.
The authors list a long string of limitations. The two that matter most: the review and compute analyses are observational, not causal, so AI share may confound with task difficulty, author experience, or review norms; and they cannot tell whether the issues spread evenly across model versions or are dragged up by earlier, weaker ones (model identifiers were masked). On external validity, the findings come from one language (C++), one monorepo, and one review-gated workflow, so transfer to other languages or organizations should be discounted. Reproducibility is also limited: the data is enterprise-internal and cannot be released, only an anonymized taxonomy and aggregated statistics are shared.
One more concern from reading it: the 11.1% warning reduction was measured on a small benchmark of 50 functions and 450 implementations, several orders of magnitude smaller than the 350k reviewable changes in production, so extrapolating it to company-wide compute savings deserves caution.