In June 2025, the CTO of a software company with several hundred engineers set a target: double engineering productivity with AI within twelve months, measured in merged pull requests per engineer. A study of what happened next, by researchers at Carnegie Mellon and Stanford, found that output per developer nearly doubled, and that the company kept up by routing work around human review: the share of pull requests with at least one human review fell from 89% to 68%.
What the study found
The paper is a preprint on arXiv, first posted on 2 July 2026 and revised on 27 October. Its authors are Hao He, Shyam Agarwal and Bogdan Vasilescu of Carnegie Mellon, Yegor Denisov-Blanch and Sanmi Koyejo of Stanford, and Pavel Azaletskiy, an independent researcher. The company, a mid-sized business software firm, is not named. It shared its data and took no part in the analysis or the writing.
- The data: 802 developers followed from January 2024 to April 2026, and 196,212 pull requests opened by people rather than bots, across 364 repositories.
- Output: pull requests per active developer nearly doubled between early 2025 and April 2026. The authors’ conservative estimate of the part due to using AI is 1.46 times, and the gain built up over months of use rather than arriving with the mandate.
- Review load: pull request volume across the company grew 3.1 times, while the number of people reviewing grew 1.5 times, so each reviewer’s load roughly doubled.
- Review coverage: the share of pull requests with at least one human review fell from 89% to 68%, while the share reviewed by an AI tool rose from about 19% to about 84%.
- Review depth: reviews with a human-written comment fell from about 39% of pull requests to about 21%, while approvals without one held at about half. The median reviewer left about three commented reviews a month throughout, while their silent approvals roughly doubled.
The authors put the review data in one sentence: the company “kept pace by routing work around human review rather than by reviewing faster”.
What it means for teams shipping AI-written code
Review is the constraint, and it does not grow with the tools. Spending on Claude Code per pull request author rose about twentyfold, from $42 to $856 a month, between July 2025 and April 2026. The pool of reviewers grew by half. Late in the period, the paper notes, the company began sending pull requests through AI review and automatic approval.
When review falls behind, nothing turns red. The merge rate in the study stayed essentially flat, so a team watching merges and output would have seen a company doing well. The authors’ warning about that number applies to any dashboard built on it: “The merge rate says little about quality, however, since once review becomes lenient or automated, the gate itself has changed.”
The target itself was a count per engineer. The authors recommend pairing such a target with other signals, because “a count that becomes a target stops measuring what it was meant to”. Why Deslopping never scores engineers makes the same argument about review.
What the evidence does not show
- A quality collapse. The authors looked for one and did not find it: the merge rate stayed flat, the share of reverts fell slightly, and fixes following pull requests labelled as AI-written rose no more than other follow-up work did. They are careful about what that means. Their measures “miss defects, incidents, and maintainability”, so the evidence “speaks against an acute quality collapse but cannot rule out quality costs”.
- That anyone stopped reading. A silent approval can follow a careful read. What the study shows is that fewer pull requests carry any trace of one, which is a narrower claim.
- An industry average. It is one company, keen on AI, with uncapped tool budgets and a public target. The authors say their estimates are “better interpreted as an upper bound than as an industry average”.
- A clean cause. Adoption was not randomised, and the size of the effect varies with the method used to estimate it, though its direction does not.
What a team can do about it
The authors’ own recommendations are the place to start: “set targets for review coverage and depth alongside throughput, decide in advance what automated review may approve without a human, and monitor what humans stop reading.” On your own repositories, that comes to four things.
- Measure coverage and depth, not only output. The share of merged pull requests with a human review, and the share with a human-written comment, both come out of GitHub’s API. Put them next to whatever output number you report, month by month.
- Write down what an AI reviewer may approve alone. Decide it before the queue decides it for you, and keep sensitive paths behind a required human review from code owners.
- Keep changes small enough to read. The study’s median pull request grew from 35 to 61 lines. Agent instructions that ask for smaller, stacked changes make a human read cheaper.
- Plan reviewing like you plan writing. If the tools have multiplied what your authors produce, ask who reviews the difference, and when.
Where Deslopping fits, and where it does not
Deslopping measures the thing this study could not: on a risky pull request, whether the person approving it can explain the lines that changed. Every pull request is scored from what it changes, and those over your threshold wait on a required check until the reviewer has answered two or three questions about specific lines. A silent approval does not pass it. Across the team, the reviewers’ ratings of the AI’s code show where it writes code people cannot follow, by module and by tool.
It does not add review capacity. A gated pull request costs a few minutes of reading, and if the queue is already the bottleneck, those minutes come from somewhere: the threshold is how you choose where. It does not review the code or find bugs. And it is no use as a per-engineer target, since it computes no per-person number at all.
The question the authors end on is the right one: “how much a human must understand before code ships”. Most teams have never written their answer down. That is the first thing to do, with us or without.
Know which risky merges someone could explain
Deslopping asks the reviewer of a risky pull request two or three questions about its lines before it merges, and shows by module and by tool where your AI’s code is hard to follow.