Vapi’s review bot puts evidence before comments

Vapi described an internal code-review bot that it says helped reduce average time from opening a pull request to merging it from more than 130 hours to 65 hours. Its October 12 account reports 1,694 reviewed pull requests over a month and 363 blocker findings across 276 of them. These are company-reported results from Vapi’s own development process.

The bot’s design is as interesting as the timing claim. It can inspect callers, run tests, or build a small reproduction before reporting a problem. Analysis and comment posting are separate jobs, and the bot can comment without approving or pushing code. That gives the team a way to use automated investigation while retaining human control of the merge.

The scarce resource is reviewer attention

An automated reviewer can produce more comments than a person can productively inspect. A system that identifies ten possible issues, only two of which matter, may slow a team despite occasionally finding a serious bug. Review quality therefore needs a measure that includes the cost of checking a false alarm.

A team trying a similar approach could classify comments by what happened next:

• Accepted and fixed, with an observable correction.
• Useful but deferred, or already known.
• Incorrect, or impossible to verify with the available evidence.

This is more informative than counting comment volume. It also creates a feedback loop for narrowing the prompt and improving the material available to the reviewer.

Ask for the smallest convincing demonstration

A useful finding explains the trigger, the observed consequence, and the affected behavior. If a change mishandles an empty result, a small failing example is often more persuasive than several paragraphs of concern. The reproduction should be understandable enough that the author can decide whether the bot has interpreted the code correctly.

Scope matters too. A reviewer may discover old problems in nearby code, but attaching every historical defect to a small change can make the pull request impossible to finish. Establish when an existing problem belongs in the current review and when it should become separate maintenance work.

Faster merging is not a complete outcome

Merge time is affected by staffing, task size, release practices, and many other factors. Vapi’s before-and-after result does not by itself prove how much of the change the bot caused. A pilot elsewhere should also track escaped defects, author interruptions, and the effort spent responding to automated comments.

The transferable idea is a reviewer that earns attention by showing its work. Automated code review becomes more useful when it produces fewer, clearer, inspectable claims and leaves the decision to merge with the people responsible for the software.