A paper turned up in my feed this week that describes my last six months more accurately than I have managed to.
Demirer, Musolff and Yang took the public GitHub histories of more than 100,000 developers and matched each one who adopted an AI coding tool against a near-identical developer from exactly a year earlier. Then they traced what happened as work moved up the production hierarchy: lines of code into commits, commits into pull requests, pull requests into projects, and projects into shipped releases.
The gains are real and they get bigger with each generation of tool. Autocomplete lifts a developer’s commits by roughly 40%. Add sync agents like Claude Code and the cumulative effect reaches about 140%. Add async agents and it hits about 180%.
Then it drains away. The same developers work on only about 50% more projects. They ship about 30% more releases. Take sync agents on their own and a sevenfold increase in lines of code arrives at the other end as a 65% increase in pull requests and a 20% increase in releases.
Those figures are from the authors’ VoxEU column in June. Worth knowing before you click through: the working paper has since been revised onto a much larger sample — more than 500,000 developers — and the numbers moved with it, to 30%, 180% and 240% on commits, attenuating to 80% on projects and the same 30% on releases. Bigger effects, bigger gap, same shape. I’m quoting the June figures because they’re the ones in the column I read.
Their explanation is Kremer’s O-ring: when production stages are strong complements, automating one of them yields bounded gains at the end, because the output still has to be reviewed, integrated, tested and released by a human. The binding constraint moves. It doesn’t disappear.
I read the 180-to-30 figure and recognised it immediately, because I have been paying the difference in cash and evenings.
My own attenuation ratio
On 7 September I had a Claude Code session run a read-only forensic across the previous thirty days of my own factory. It counted 306 pull requests. Over the same window, the review layer ran 1,377 deduplicated rounds. That’s 4.4 review rounds per candidate, and 99.5% of them were attended, which is a polite way of saying I was sitting at the desk while they happened.
The API bill for those rounds was US$783.36.
That is the paper’s Figure 1 with an invoice attached. The code got written faster every month for a year. The number of things I actually shipped moved much less, and the gap showed up as a review queue that I personally sat in front of.
The stage generates its own work
Here is the part the paper doesn’t have the data to see.
The same forensic pulled 164 late-arriving P1 findings — the serious ones that surfaced at round eight or later — and classified where each defect was born. 6.1% pre-existed the work. 37.2% were in the original code and had survived seven or more rounds of the same reviewer looking straight at them; one file took 64. And 54.9% were created by the review-fix commits themselves. The remaining 1.8% is three findings: two that only existed because the work had grown past its original diff, and one nobody could place.
More than half of the serious late findings in my review layer were defects that the review layer had caused.
That’s not a fluke of one branch. Back in August I ran a controlled comparison between a cheap-model review loop and a frontier gate, and measured self-inflicted findings at 8% for the loop and 45% for the gate. The more capable the reviewer, the more work it made for itself to find.
So the downstream stage isn’t only slow. It’s reflexive. Point enough compute at it and it will happily keep itself busy, and every round of that has a human at the end deciding whether the finding is real.
Eight draws, one catch
The paper ends on an optimistic note: tool vendors are actively working to shift the bottleneck, for instance by promoting AI tools for code review. Reasonable. That’s exactly the thing I spent August building.
On 8 and 9 September I tested it properly. I took a real defect the old system had caught three days earlier — a file untouched by the diff, whose contract the diff had quietly broken — froze the exact commit, and replayed it past every reviewer I could point at it.
Hosted Codex review on GitHub, twice: found a different, legitimate P1 in the diff, never reached the untouched file. The in-session Codex plugin: a clean review, no findings at all. The fork’s own adversarial gate prompt, run through the Codex CLI on two different models: nothing either time. Three more draws on the model that originally caught it: nothing, three for three.
Eight draws. Zero reproductions of the original catch, which came from a live lane I cannot replay. Gate rounds are stateless, so the honest reading is that the catch was recall variance on a hard class, not a capability my setup possesses.
The draws weren’t empty-handed, though, and that turns out to be the interesting part. They kept finding other real things — one of them a live P1 in my current verification doctrine, which is now its own ticket. Different reviewers find different defects. The same reviewer, run again, mostly finds its own last answer.
The class I was chasing turned out to be docs-as-contract: three markdown files had changed a verdict vocabulary that a Node script enforces, so the documentation and the code now disagreed. Every single-pass reviewer read prose as prose. The fix is a parity test asserting the enum in both places — it now runs in CI, takes milliseconds, and cannot miss. No language model was ever going to be the right instrument for that.
Which is the useful lesson from the experiment, and not the one I wanted. Throwing a better reviewer at the review bottleneck mostly buys you more findings about the reviewer.
The bottleneck nobody measures
Review at least leaves a trace. The one that’s been eating my week doesn’t.
Claude Code stops at a decision seam — a schema call, an implementation fork — and offers three or four options with a recommendation. I usually can’t adjudicate it myself, so I copy the question into a chat window, get an answer from a model with no stake in the last forty minutes of reasoning, and copy the answer back. Two capable agents, and a courier in the middle.
That’s filed as VAS-2818 now. Deep specs don’t help if I’m still the ferry captain. It doesn’t appear in anyone’s productivity statistics, because from the outside it looks like a developer typing.
What I actually did
On 7 September I stopped building instruments and rewrote the review layer around what it’s for.
One discovery pass per frozen commit, not a loop. Reviewers are read-only and never edit the artefact they reviewed, so the 54.9% class has nowhere to come from. Fixes land as a new commit and get one re-review that checks the named findings only. Unattended work ends at a parked pull request, and a human merge stays terminal.
The instrument pile went with it: about 5,400 lines of runaway breakers, harvest gates and their test suites deleted from the fleet config, plus another thousand of the doctrine that justified them, each deletion landing in the pull request that shipped its replacement. The 9,000-line fork those instruments hung off is scheduled to follow and hasn’t yet — I’m not going to tell you it’s gone while it’s still sitting on my disk. I had built a tiered review system that was 20 times cheaper per round, and then run so many rounds that cheapness stopped mattering.
It’s four days old. The pre-registered read is 21 September: review events per merged pull request at a median of four or fewer, no API-billed frontier rounds fired from inside a session, fix-induced findings under 15%. If those don’t land I’ll write that up too.
The thing I’d take from the paper is not that the gains are fake. A 30% lift in shipped releases would be a remarkable result for any workplace technology. It’s that the gains stop being about code generation almost immediately, and start being about whichever stage still has a person standing in it. Mine is a person who was, until very recently, reading 1,377 review rounds a month and ferrying questions between two robots.
Automating the writing was the easy part. It always was.