How Aviator reviews 4x more AI code without reading every line

Aviator says the same nine engineers merged 4x more pull requests after changing code review from reading raw diffs to verifying intent, acceptance criteria, invariants, decisions, and concrete evidence. Median time to merge fell from 2.1 days to 3.4 hours, reverts dropped by 40%, and a human still signs off on every pull request.

The problem: code review does not scale with AI

The team's code flipped from human-written with AI help to AI-generated with human help. Weekly merged pull requests went from about 100 to more than 400, and the median AI-generated pull request reached 640 lines, past 400 lines, which is widely considered the limit where bug detection falls off.

Catching bugs was never the main point of code review anyway. In Microsoft's 2013 study by Bacchelli and Bird, only 14% of review comments were about defects. Review is how engineers share knowledge and keep their mental model of the system current, and that is the part worth protecting. When agents write most of the code, it is also the part most at risk. Asked whether they simply stopped reviewing, the team says no: they stopped reviewing the diff. Their conclusion is that decisions need a person, and a diff is the worst place to make them.

What changed: verify intent, not lines

Code review used to do several jobs at once: catch bugs, enforce team conventions, settle design decisions, and keep everyone's understanding of the system current. The team split those jobs up and gave each one a better home.

Intent and acceptance criteria are captured from the coding session with the agent, not written as an up-front spec. Every time an engineer corrects the agent or answers its question, a criterion is created. The team says it used to throw those away when the session ended. Intent answers why the change is made; acceptance criteria answer how it should behave. Intent without criteria cannot be checked, and criteria without intent cannot be judged.

That second half is exactly what a reviewer reading the diff would never check. Not every criterion needs a browser: an API contract is proven by a recorded request and response, and a code rule by an ast-grep match or a file-and-line citation. The advice is to pick the cheapest proof that settles the question. A green CI check only says the tests you thought of passed.

Decisions come from three places: the coding session, the agents' argument, and failed verification runs. The pull request lists every one, where it came from, and who settled it. On a pull request that added retries for failed webhook deliveries, four of five decisions were settled; the agents disagreed on whether to deduplicate retried deliveries, so a person settled that one.

Engineers stop reading every line. They own the rules that check every line, and keep their understanding of the system by reading intent, evidence, and decisions.

The results

Review work did not disappear. It moved to refining invariants, debating decisions, and improving verification. The transition can hurt: month one is double work, by month two invariants catch up enough that reviews shrink, and by month three people read intent and evidence, not code.

Sources: The New Stack