跪拜 Guibai
← All articles
AI Programming · Claude

How the Claude Code Team Verifies the 90% of Code That Claude Writes

By ZzT ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Verification is the difference between running one Claude at a time and running dozens in parallel. This workflow shows how to make an AI reviewer trustworthy by constraining where rules live, capping reruns, and measuring recall in stages; without these controls, AI-written code can pass tests while the product breaks.

Summary

Anthropic's Claude Code team no longer reads every diff line. Instead, it defines what "done" means, writes checks that run the app, and inspects the evidence. The team has Claude verify changes by actually starting and operating the application through a skill named verify, which runs before every commit. Claude Code now ships a built-in /verify command and, since v2.1.286, the commit instruction enforces this pre-commit run unless the change is docs-only or tests-only.

The team keeps its reviewer honest by placing rules on branches the changes cannot modify. A 39-item reliability checklist, built from past incidents, tells Claude which risks to check for a given diff, typically 3 to 10 items. If a check fails three consecutive times, Claude hands off to a human with the reason stated. REVIEW.md on the protected default branch customizes the managed code review, while local /code-review reads CLAUDE.md instead. Commit gates start in observe mode to record would-be blocks before enforcing, with an escape hatch after two blocks.

The team evaluates its reviewer by measuring recall in three stages: raw discovery, after verification, and final review. Missed bugs become rules via pattern detection or replay of known incidents. Two numbers in the post are not the team's own: "up to 10 percentage points" comes from a Managed Agents test, and "two to three times" is Boris Cherny's personal estimate. The original post gives no effect numbers for its own verification stack.

Takeaways
— Anthropic's Claude Code team writes and reviews about 90% of its code with Claude; humans define "done" and check evidence instead of reading every diff.
— The team's verification is running the app, not just tests; a verify skill starts and operates the app, and Claude runs it before commit.
— Three test categories are recommended: end-to-end (preferred), property (for edge cases), and differential (for migrations with unchanged behavior).
— A 39-item reliability checklist is built from past incidents; each item has a diff-detectable trigger, a check, and an incident or PR link.
— Claude attempts to fix a failed check itself; after three consecutive failures it hands off to a human.
— REVIEW.md on the protected default branch customizes the code review service; local /code-review reads CLAUDE.md instead.
— Review results are only valid for one commit; a new commit invalidates them.
— Cheap checks run first: Claude runs tests, build, and linter before opening a PR, enforced by a Stop hook or /goal.
— The commit gate has off, observe, and enforce modes; observe records would-be blocks, and two blocks in a session allow the next commit.
— Reviewer recall is measured in three stages: raw discovery, after verification, and final review.
— Missed bugs become rules via pattern detection (two PRs, three agents attempt to disprove) or replay of known incidents.
— Flaky tests are strictly handled: no skipping, no sleep, no retry until pass; delete if root cause not found.
— Two numbers in the post are not from the Claude Code team: "up to 10 percentage points" from a Managed Agents test, "two to three times" is Boris Cherny's estimate.
Conclusions

The insistence that checks live on branches the change cannot modify is the core difference from typical agentic coding setups: an AI that can edit its own rubric will grade itself leniently.

Measuring recall in three stages rather than one end-to-end score is a smart move: it surfaces whether the system loses already-found bugs in the middle, which a single number would hide.

The commit gate's observe mode treats a process change like a code change with a rollout, letting a team validate false positives before enforcing.

The 39-item checklist format (trigger, check, incident) makes past knowledge executable for an agent without requiring humans to remember incidents.

Concepts & terms
verify skill
A runnable skill in Claude Code that starts and operates the app (via browser, simulator, or API) so Claude can confirm a change works; it's stored under .claude/skills/ and run before commit.
REVIEW.md
A markdown file placed on the protected default branch that configures what the managed code review reports (severity levels, excluded paths, evidence requirements); it cannot be modified by the PR itself.
Commit gate
A mechanism that blocks a commit if the session hasn't run /code-review or /verify; it has off, observe, and enforce modes, with a two-block escape hatch.
Property testing
A testing style that generates many random inputs to try to disprove an invariant, catching edge cases that unit tests miss; tools include fast-check, Hypothesis, proptest.
Differential testing
Running the same inputs through old and new systems to find behavior differences; used for migrations where behavior should stay unchanged.
Stop hook
A Claude Code hook that triggers at the end of each Claude reply and can block or cause continuation; used to enforce tests before PR, with a block cap and environment override.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗