One Team's AI Code Review Pipeline: How to Let AI Review AI and Actually Trust It
This is a concrete blueprint for turning AI code review from a one-off suggestion box into a self-improving, measured part of the delivery pipeline, showing how a small team can operationalize AI review with commodity tools and without continuous human babysitting.
The system uses Alibaba's open-source Open Code Review (OCR) as its engine and a cheap internal model, betting that rule quality matters more than model horsepower. Rules are split into a team-level baseline stored in a shared git repo and per-project rules in each repo's root; both are merged and versioned, with path matching, three severity levels, and exceptions to reduce false positives.
A shared CI template runs the review job in parallel with the build, tolerating failure and using smart diff baselines (MR, push, or new-branch fetch). Results go three ways: GitLab comments anchored to exact lines, Feishu cards color-coded by severity with per-author splitting, and real-time ingestion into a metrics store.
A reception layer identifies AI comments by format, dual-channels ingestion (real-time + reconciliation), and infers status from developer replies (fixed, false positive) to feed a dashboard. The false-positive rate becomes the input for rule optimization. Finally, the coding AI itself consumes the findings: it fixes redlines locally, replies to CI comments with fixed/obsolete/false-positive justifications, and those replies flow back into the dashboard. In one test, 14 of 17 comments were fixed, 2 obsolete, 1 false positive.
Treating review rules as versioned, mergeable artifacts alongside code turns AI review into a controllable, testable system rather than black-box prompt magic.
The false-positive loop is the strongest part: users vote with their feet, and the metric directly drives rule changes, which is a pragmatic way to operationalize AI quality.
Handing the review conclusions back to the code-writing AI closes a loop that most teams stop at distribution; this is what moves humans from reviewing every comment to surfacing exceptions.
The reliance on Feishu as a notification channel is China-specific; Western equivalents would need Slack or Teams, but the pattern is identical.