How to Use AI Coding in Systems Where You Can't Afford Mistakes? Distributed Storage Practices and Boundaries

AI can make code writing faster, but in a "cannot fail" distributed storage system, how do you ensure that efficiency gains don't turn into risks? This article combines Meituan's foundational technology department's real practices in AI review, problem troubleshooting, and automated development to share what AI can help with, where it easily makes mistakes, and where humans still need to be in charge. More important than "writing more code" is making code and conclusions verifiable. We hope these experiences provide a reference for AI applications in complex systems, safeguarding quality and stability while enjoying efficiency gains.

1. Background: Why Distributed Storage Needs a More Careful Approach to AI Coding

Since the beginning of this year, with the proliferation of agents, AI coding has become usable in feature development, test generation, script writing and execution, and problem troubleshooting: reading code is faster, implementation and tests can be first written by AI, and some of the chores outside development can be handed to it.

Efficiency is up, but the quality ledger needs to be recalculated. With the same headcount, a release can deliver more features and change more code. Even if AI-written code has the same per-thousand-line defect rate as human-written code, the absolute number of defects introduced in a release will still rise. So the goal of AI coding cannot be just faster code writing. While speeding up, code quality and verification sufficiency must be higher than regular development requirements.

Distributed storage amplifies this problem. The codebase is large, module relationships are complex, and consistency, concurrency, persistence, failure recovery, and performance all come into play. A seemingly local change might be fine in normal operation, but expose problems only when a node restarts, a disk acts up, the network jitters, or under high concurrency. When problems occur, the impact isn't limited to one feature; it affects data availability and system stability of upstream businesses, and in severe cases, data correctness. Storage services support a large number of upstream businesses; faults have a broad impact, are difficult to fix, and have an extremely low tolerance for defects.

To meet such quality requirements, generating code with AI isn't enough. AI review (Code Review), AI testing and validation, AI Debug, and controlled automation also need to be built up so that AI-generated code can safely enter the development process. This article combines the storage team's practices in AI review, AI Debug, and AI automation to discuss how, in a system as complex and low-tolerance as distributed storage, one can use AI to speed up while relying on stricter review, testing, debugging, and automated validation to protect stability and data correctness. We hope it also serves as a reference for other similar systems adopting AI coding.

2. AI Review (Code Review): A Useful Supplement, but Not a Substitute for Human Judgment on Design Context

2.1 Where AI Review Excels

1. Catches cross-file inconsistencies

Example: ENABLE_UNITTEST controls the compile branch of test mock variants at three call sites, but the CMake migration renamed this to WITH_UNITTEST, and the old name was never assigned again. Looking at any single file, you can't see the problem; the bug exists only in the gap between files.

2. Builds executable evidence

Example: AI pointed out "no boundary check here" and didn't stop there. It then constructed a malformed packet declaring 1 GiB in value but with only a 5-byte actual body, hit the real decode entry, and captured the crash.

3. Admits what it can't prove

Example: For a suspected concurrent race, AI could only walk through the code to find a suspicious path. It explicitly stated that proving the interleaving would require ThreadSanitizer, which couldn't run in the current toolchain, so it marked this as "unverified" rather than classifying it as a defect.

4. High recognition rate for some common races

Example: An update that read-checked first, then wrote and advanced, with no mutex in between, was consistently identified in multiple reviews, and the suggested fix direction was also consistent (merge into a single atomic operation). This class of common concurrency patterns is recognized reliably and can be turned into fixed check items.

5. Doesn't miss deleted code in PRs

Example: A refactor merged an interface into a single RPC and conveniently deleted the old logic that cleared the local cache after success. AI pointed out this leaves a short time window during which stale data could be read. Humans are least sensitive to deleted lines.

6. Reviews incremental changes in the context of existing code

Example: Before reviewing, AI first analyzes the existing codebase. In a large PR, new code might incidentally fix an existing problem; human review tends to report such a side fix as a new issue. AI, instead, lists "already existed, fixed incidentally this time" as a separate item and declares it won't be reported as an issue.

7. Review isn't limited to code-level changes

Example: In a PR, build files from three repositories referenced each other using feature branch names without pinning versions. If merge order is wrong or branches are cleaned up, the main build breaks. This is a "merge orchestration" problem, which is exactly what people looking only at the PR page are most likely to miss. AI treated the build files as review targets too.

2.2 Where AI Review Goes Wrong

Most of AI's misjudgments arise because it can't see things beyond the code: operational processes, design decisions, and team conventions. Our practices also revealed problems in AI review around fix suggestions, signal-to-noise ratio, severity classification, and evidence selection.

1. Code reachable ≠ operationally reachable

Example: The report marked a code path with an expired epoch as a real risk. Human re-review didn't question the code itself; it questioned whether surrounding systems could produce input that would reach that branch: "EPOCH_NOT_MATCH only appears after migration split +1. After the split, the old region no longer exists. If you migrate again, the src also doesn't exist. So EPOCH_NOT_MATCH theoretically can't reach this branch."

The same situation appears in a report flagging a one-day expiration time as a race window: "Migration is a manual operation. Online, a migration that started a day ago and still hasn't succeeded wouldn't exist; human intervention is needed. A one-day expiration is precisely to address that." AI can model the timer, but it can't model the human process wrapped around it.

2. Treats deliberate deviations from common practice as defects

Example: The report flagged a consistency check that directly calls abort() in production as a correctness risk and suggested changing it to log an Error rather than crashing. We chose abort because storage systems prioritize correctness over availability. In specific scenarios, we'd rather abort than let the request continue and produce dirty data that is extremely hard to handle.

3. AI's attached fix suggestions also need review

Example: For a 32-bit overflow, AI's proposed fix was to check whether the declared size matches the actual bytes received. On review, we found get_size() returns an int, so a size between 2^31 and 2^32 gets truncated to a negative number, and the check itself silently fails. The individual case was fixed, but the class of problems wasn't.

4. Cheap "please confirm" items crowd out real signals

Example: AI review marked thread pool defaults, renamed config items, and a removed second-level cache refresh switch as "to be confirmed." The eventual reply for each was based on the team's own historical load-test data, unrelated to code defects: "Adjusted based on load tests; reducing background thread count lowers impact on foreground."

5. Severity classification is distorted

AI lacks a clear quality threshold: the same class of theoretically possible problem might be flagged as highest severity one time and unimportant the next. When you get an AI report, you can't fully trust the severity classification. To be safe, you end up reviewing every item individually, making it hard to focus on truly high-severity problems.

6. Treats comments, header-file assertions, and design documents as evidence

Example: Two AI reviews used the same "scenario × constraint" framework and reached opposite conclusions. One directly quoted a header-file assertion to judge the implementation correct; the other verified the concrete implementation and judged it wrong. Another example: simply relying on a design document—the version continuity check required by the migration design doc—had no reference at all in the code.

The two reviews looked at the same code; the difference was whether they verified these "explanations" themselves.

2.3 Making Review Conclusions Executable: Build Tests, Integration Tests, and Runtime Logs

Static review can only say "this path exists in the code." Whether that path will actually error, whether existing protections can stop it, and whether it has happened in production require three additional kinds of evidence.

2.3.1 Build and Unit Tests: Turning Reasoning into Red-Green

Experiment: AI took 117 suspected issues from a full-repo review of a project and wrote a gtest for each. A test failing on master counted as confirmation. Results: 36 red tests confirmed, 3 falsified (false positives), 41 could only be confirmed via source code, and 37 unverified.

• Definitive verdicts. The turning-red tests clustered in three categories: input triggering abort, errors swallowed yet still reported as success, and decode out-of-bounds. Severity can be determined by the failure symptom, no need to rely on AI's phrasing.
• Can falsify. 3 findings were overturned by tests, something static review rarely achieves.
• Exposes CI problems. For example, the test report hardcoded failures="0", and failing cases weren't included in the summary. So AI must build and read raw output, not just trust CI's green light.
• Boundaries. Races and use-after-free need sanitizers; timing issues need scheduling control. Fault injection points, restricted hosts, clock skew, and other conditions also have gaps. The "unverified" list is essentially a to-do list for test infrastructure. Red tests also only prove the code can error, not that the input will occur in production. In one data migration scenario, tests confirmed a data-expiration issue, but human review found that in the manual operation flow, the expiration was deliberately designed.

2.3.2 Runtime-Near Integration Tests: Main Value Is Reducing False Positives

For example, one project's class implementation had special considerations so we could spin up a real Raft multi-node cluster in a single process, covering all scenarios that require a "production cluster."

• Let existing protections participate in verification. An AI-identified issue claimed leader switch would cause data loss on delete, but an existing integration test happened to switch the leader during a delete window and proved it safe.
• Adjudicate disagreements. For "can a snapshot capture a split intermediate state," two AI reviews disagreed. Writing a test that goes through the real change_peer entry with injected synchronization points can directly adjudicate.

The boundary of fake/mock is the boundary of verification. Fakes must go through real entry points, use production config, and document what was replaced.

Example: A distributed KV hang-alert bug wasn't caught because the test setup differed from what actually happened in production; a network-phase drop protection was hand-fed a timestamp in the unit test, but the production path never sets it.

2.3.3 Runtime Logs: From "Could Happen" to "Has Happened"

Experiment: We gave a storage node's 11 days of WARN logs (678,000 lines, 34 call sites) to AI to map back to source code. It found 5 defects, all confirmed by the responsible engineers, with zero false positives.

• Finds issues static review won't suspect. Example: "fail times 1" appeared only once in 11 days, indicating a counter was stuck and the hang alert would never fire.
• Falsifies wrong assumptions. Example: A 19-minute response had a ratio of summed stage times to total time consistently 1.0, ruling out clock errors, and leading to the discovery that drop protection was missing in the send phase.
• Provides a basis for severity. Example: A measured stage time hit 1,156 seconds, less than 2x the int32 overflow boundary for that storage duration.

Two of the defects weren't directly triggered by the logs but were found when AI read adjacent code to explain the logs. They were also real. Logs' real config and magnitudes also make adjacent-code reasoning more accurate. This is a single sample and doesn't prove zero false positives always, but it supports a practice: when runtime logs are available, attach them to both review and debug. Logs' limitation is they only cover executed paths, and line numbers may not match the deployed version (locate by message text).

2.3.4 Summary

Evidence	Answers	Can't Answer
Static reasoning	Whether this path exists in the code	Whether it actually errors, is reachable, or happens online
Build + unit tests	Whether it errors under given inputs	Races and timing; whether the input will occur online
Integration tests	Whether it's still reachable under real interactions and existing protections	Whether it actually happens online
Runtime logs	Whether it happens online, frequency and magnitude	Paths never executed

These four evidence layers complement each other: static review finds candidates, tests judge truth, logs show whether it happens in reality.

2.4 Recommendations for Using AI Review

1. Output conclusions in four evidence tiers, not mixed: test-confirmed (with a test that fails on current master), test-falsified, source-traced confirmed (explain why it can't be written as a test), unverified (specify the exact missing conditions, not a generic "needs environment"). Aggregating "unverified" items across reviews gives you a to-do list for test infrastructure. Reports should also distinguish whether each conclusion comes from tests, runtime logs, or reading code. Locate logs by message text and function name, not line number.
2. "No problem" conclusions must list the paths checked: e.g., "the following N paths have been checked, with line numbers and why counterexamples don't hold; the rest are not covered." Under the same scenario framework, a weaker model configuration gave "no correctness issues," while a stronger configuration derived an opposite causal chain. "No problem" stops subsequent investigation, making it the most expensive sentence in a review.
3. Review the checked-out code, not the PR web page: any assertion about repository state—what's managed, what's committed, whether there are conflicts—must include the commands and outputs actually run on the real branch. For example, a PR's web diff showed only about 2% of the real changes; the server-side merge preview dropped most of the branch content. So before reviewing, verify the review target itself with a three-way diff.
4. For AI's type- or boundary-related fix suggestions, list all types in the call chain first: before adopting, list the signature of every function the value passes through. Width mismatches often appear two functions downstream (see 2.2 #3).
5. Context: give stable invariants, not all documentation: several misjudgments above trace to AI missing background: which invariants are guarded by deliberate aborts, which timers sit behind manual processes, and which designs were already settled. But these details are numerous and implementations change often; outdated information misleads AI (we tried feeding project docs to AI and using historical project docs as a knowledge base; both were skewed by outdated content). The more effective approach is to provide only stable, high-level invariants and key design constraints, maintained alongside the code. Design documents for the specific PR work well with prompts. A defect knowledge base (including accepted/rejected/deferred defects) prevents repeated false positives (be wary it can cause regressions to go undetected). After each review round, add "which missing context caused AI to misunderstand," not simply pile findings into agents.md/claude.md. The knowledge base's effect also depends heavily on model capability: SOTA models can distinguish redundant info, weaker models are easily misled.
6. Don't use multi-agent voting instead of correctness judgment: multiple agents read the same diff and lack the same background; their blind spots are shared and they'll be consistently wrong at the same place. In practice, merged reports have taken multiple agents' shared wrong conclusion as correct and marked it highest priority. Votes reflect overlap, not correctness. Cross-review conclusions still need item-by-item verification.
7. Require AI to declare whether execution degraded: AI usually does a code index before review; if any step fails, it might downgrade to grep-style tactics and continue. Accuracy drops significantly, but the report may not mention it. So require the report to list which tools and steps were actually used and which steps failed or degraded, to judge conclusion trustworthiness.

3. AI Debug: Speeds Up Location, but Cannot Directly Serve as Root-Cause Conclusion

AI Debug's most direct value is shortening the distance from "seeing an anomaly" to "finding a direction worth verifying." It quickly correlates logs, monitors, call stacks, code, and config, proposes candidates in multiple directions, and can generate diagnostic scripts, minimal reproductions, and comparison experiments. For cross-module, cross-repo problems, it saves a lot on information assembly and repeated investigation.

But AI's explanations, however complete and specific they look, cannot be taken directly as root-cause conclusions. Distributed storage faults often depend on concurrency timing, node roles, persistence state, recovery processes, external dependencies, and operational actions. AI usually only gets a slice of that. It can easily find a code path that "explains the phenomenon," but it hasn't proven that path was actually taken online, nor that no other cause could produce the same result.

So a better positioning is to treat AI as a hypothesis generator and verification assistant: it expands the search space, compresses the investigation space, and executes controlled verifications; humans supply system semantics, judge evidence strength, confirm root cause, and review fixes.

3.1 Where AI Debug Excels

1. Quickly assembles scattered clues

AI can align logs from different nodes, monitor metrics, call stacks, config, and code changes onto a single timeline, then hunt for the producer and consumer of the same state. Cross-file search, comparing normal and abnormal paths, and version-diff assembly are mechanical work best left to it.

Example: Creation succeeded, subsequent operations failed, and a later retry recovered. AI connected three code repositories and found the error came from "resource just created not being found." After humans added "the database already has the record, the request went to the right place, and the retry succeeded later," the investigation converged on cache: before creation, one machine had cached "resource doesn't exist." After creation, that cache wasn't updated promptly, so subsequent operations failed. After periodic refresh, the cache aligned with the database and the retry succeeded.

2. Proposes verifiable candidates in parallel

The same latency, error code, or status anomaly often has several explanations. AI can expand from cache, queue, concurrency, network, persistence, redo, and resource contention directions, and for each candidate list necessary conditions and missing evidence. This candidate set is to help engineers decide what to investigate next, not for AI to pick "the most likely root cause" from it.

Example: Old data never cleaned, space growing continuously. AI laid out candidates like "cleanup task didn't run," "data still in use," "deletion failed," and pointed to corresponding log clues. Humans then found: a node had started working, but the "startup complete" flag of the cleanup module wasn't set, so every round of cleanup was skipped. AI provided a checklist of items to check individually, and humans used live logs to confirm which one actually hit.

3. Turns local mechanisms into executable experiments

AI can turn code conditions, thresholds, and execution order into explicit experiment expectations, then check with actual inputs and config whether these conditions can hold simultaneously. Experiments should go through real entry points, let the system produce the target state itself, and set up contrasts and assertions that distinguish between explanations. This turns "apparently possible" into "reproducible or falsifiable."

Example: Turning "where the space is lost" into a full-link experiment. When a request's validation failed, AI used a single-point experiment to confirm the server strips leading/trailing spaces, and initially attributed the fault to the server. When humans asked to check the client too, AI turned the problem into a testable contrast: record the content the client uses to compute the signature, what's actually sent, and what the server receives and uses to recompute the signature, then compare segment by segment to find where the first difference appears. If the content changed before sending, the "server first rewrites" explanation is negated. The experiment found the client first computed the signature on the space-containing content, then stripped spaces before sending, making the signature inconsistent with the actual payload. AI then corrected its original causal attribution.

4. Continuously narrows the problem space with new evidence

AI can use staged observations and contrast experiments to progressively confirm which factors participate and adjust investigation priority. After each intervention, compare results to the original anomaly and the normal baseline, see what's explained and how much deviation remains. If metrics haven't returned to baseline, keep investigating; don't treat partial improvement as the end.

Example: See 3.1 #1. "The database already has the record" ruled out the explanation that creation didn't write data. "The request went to the right place" ruled out sending to the wrong location. "The retry succeeded later" pointed to a cache state that changes over time. Each new piece of evidence narrowed the candidate set, and combined with cache update logic, explained the whole process.

3.2 Where AI Debug Goes Wrong

1. Incomplete input, missing evidence, yet a complete and certain conclusion

Code and logs usually can't contain all the online state, design decisions, and operational constraints. AI tends to fill information gaps with unverified premises, write partial reasoning as a complete causal chain, and even reach definitive conclusions without log, code, or field evidence. Require it to explicitly list dependent assumptions and missing information, attach checkable evidence to every key judgment, and state which step each evidence supports. A conclusion with no evidence can only be labeled as assumption or to-be-verified. An analysis that reads smoothly and includes code line numbers only means it's easy to review, not that the conclusion is correct.

Example 1: A partial query missing isn't the same as IDs repeating. AI found that a query before creating an object only returns normal objects, not those being deleted. It treated this query as the key guard against duplicate creation and inferred "if the old object can't be found, the same number could be used to create again." Humans pointed out that new object numbers are guaranteed unique by a centralized allocator; this query isn't responsible for number allocation. The premise "another creation flow with the same number could enter" didn't hold, so the risk conclusion was withdrawn.

Example 2: Parallel-test failure wasn't the initially guessed name collision. When tests were changed to run simultaneously and started failing, AI initially attributed it to random name collisions. After humans asked for log evidence, the analysis confirmed that one test, when cleaning up resources, also deleted another test's resources. Name collision sounds plausible but doesn't replace evidence of the actual deletion behavior.

2. Treats a single-point fact as the causal location

A single-point experiment can prove a component exhibits some behavior, but not that this anomaly originated there. Trace along the actual processing chain, align inputs, outputs, and states at each step, find the first point that deviates from the expected path, then verify how downstream steps propagated or masked the deviation. Only when upstream and downstream evidence is complete can you distinguish where the anomaly was generated versus where it was observed.

Example: See the space-validation issue in 3.1 #3. "The server strips leading/trailing spaces" was indeed supported by a single-point experiment, but it can't prove this request's space loss originated at the server. The full-link experiment found the difference existed before sending, showing that real evidence doesn't guarantee correct causal placement.

3. Mixes contributing factors, mitigation effects, and root cause

If a phenomenon improves after an action, that only shows correlation; the action may have changed multiple conditions or destroyed the scene. The cause of the initial error, the condition that made an existing problem manifest, the factor that amplifies impact, and the measure that restores service—these must be identified separately. Use controlled variables and before/after evidence to confirm each one's role; don't promote "effective mitigation" to "root cause found."

Example: See the creation-then-operation-failure in 3.1 #1. This request recovered because the cache refreshed, but the defect of not updating the old cache on resource creation still exists, and the same operation sequence can fail again. Explaining "why this recovers" is not solving "why it happens."

4. Static reachability or green tests mistaken for online occurrence

Static analysis finding a dangerous path is just a risk candidate. To judge whether it causes real problems, check whether trigger conditions can hold simultaneously in real operation, and whether existing system protections can stop the error or limit impact. To further claim it's the cause of this incident, you need field evidence that the path was actually taken and the observed anomaly was caused by it. Test passing only proves the behavior covered by the given environment, inputs, and assertions; it doesn't say the same problem occurred in production, nor that the system is correct overall. Static reasoning, experimental verification, and field evidence each prove different things; label which step each supports.

Example: The ID-conflict false positive in 3.2 #1. AI derived a duplicate-creation risk from a partial query omission, without first proving the same ID could be allocated again. The global number rule makes that premise false, so the derivation can't prove actual risk reachability, let alone be written as an occurred incident. This case illustrates static reasoning's proof boundary.

5. Makes local optimal patches around trigger cases, doesn't thoroughly solve similar problems

AI tends to give the fastest change near the failing line to make the case green, without checking whether the same constraint is violated in other scenarios—especially resource sharing, concurrent operations, and data creation, use, recovery, and cleanup.

Example: The local plan needs to cover concurrency, replicas, and recovery. In a data-split scheme review, AI defaulted that data versions remain unchanged across steps and once relied on the current machine's cache for later initialization. When humans asked whether background tasks could change versions, how other replicas or restarts would obtain the same info, the scheme added version protection and wrote necessary info to a recoverable log, keeping cache only for acceleration. This is a scheme gap caught in design review, not an occurred online fault.

3.3 Recommendations for AI Debug: Recommended Working Mode

The core principle: save facts first, then hypothesize; verify causality with experiments, and have humans review root cause and fixes; if the phenomenon isn't explained yet, keep investigating.

1. Fix the field facts and record state-collection boundaries

Before analysis, build a fact sheet without speculation, including fault manifestation and impact, incident sequence, running environment, and operation history. Then supplement data, cache, external dependencies, and config effect as needed. Raw logs and monitoring, business stakeholders' accounts, and static code reasoning should be recorded separately. Each state must note collection time, source instance, and read method. When you can't confirm several pieces of evidence are from the same moment or the same request, say so explicitly; don't stitch them into a "simultaneous" scene. Operations like restart, retry, and cache refresh change the scene; save evidence before doing them.

2. Have AI output candidate sets, don't accept a single root cause

For each candidate, have AI write trigger conditions, code path, state changes, what it can and can't explain, current evidence, and falsification method. Humans then add global invariants, historical design decisions, and operational constraints—like who allocates IDs, how roles switch, which states persist, and which failures allow retry.

Use the following names consistently during investigation to keep conclusions from being silently upgraded in wording:

• Observed fact: directly supported by logs, monitoring, field state, raw bytes, or test results;
• Candidate: can explain part of the phenomenon, but necessary conditions or the causal chain aren't closed;
• Verified factor: experiments show it participates in the problem, but it can't yet be classified as root cause;
• Root cause: first error, propagation process, protection failure, and final user result are closed, and major alternative explanations have been excluded;
• Amplifying factor: expands impact or prolongs the fault, but doesn't create the initial anomaly;
• Mitigation: can contain damage or restore service, but doesn't eliminate root cause;
• Unverified risk: statically possible, but hasn't been shown to be reachable online or cause impact.

This candidate set and classification is a continuously updated investigation map: reclassify as new evidence arrives, exclude falsified candidates with the basis recorded, and don't upgrade factors that only explain part of the phenomenon to root cause.

3. Use tests to reproduce, but define what they prove

Unit tests are good for local algorithms, boundary conditions, and a specific minimal timing, but they can't prove a production request will take that path. End-to-end reproduction in a test environment can validate propagation chains across components, but only that part matching production—versions, config, data state, and entry points—can be extrapolated to production.

For concurrency, timeout, partial failure, and recovery problems, prefer fault injection to control trigger timing, and observe both business outcome and intermediate states. For key verifications, complete the red-green loop as required in Section 5.

4. Follow the full chain to find the first error

For each high-priority candidate, record actual state along "production—propagation—persistence—commit—consumption—recovery—cleanup," find the first divergence from the correct path, then explain how it became the user-facing result step by step.

5. Have humans confirm root cause and review fixes from global invariants

Before AI's conclusion goes into an incident report or drives a fix, an engineer familiar with the module must review: whether field evidence proves the necessary conditions, whether the chain from first error to user result is closed, why protections failed, and whether major alternatives are excluded. AI's "high confidence" doesn't replace this gate.

Fix review should start from the system's key design and global invariants, solving this class of problem rather than just making the trigger case green: check along relevant call chains for locks and references, roles and epochs, persistence and replay, error propagation, and the normal, abnormal, recovery, and cleanup paths. Evaluate compatibility, performance, and release risk. Changes to both production and test code must be reviewed.

4. AI Automation: Good at Verifiable Tasks, but Key Operations Must Be Controlled

The team experimented with using AI for automation, having it continuously handle environment checks, build modifications, failure localization, and test regression—tedious but relatively pattern-fixed work. The team used AI to initiate two such tasks: 1) Distributed KV build system upgrade. The project originally used Make + Autotools, which was a pain to maintain, so it needed to migrate to CMake; 2) Distributed KV server adaptation for ARM servers. The server needed to run on Huawei Kunpeng servers to reduce costs, requiring ARM adaptation. Both tasks had clear implementation paths and verification methods, but involved many details and heavy workloads. Manual handling required repeated editing, execution, and checking. In practice, AI did save significant repetitive labor and could continue iterating based on build and test feedback. But over long continuous runs, it can also go down wrong paths, expand the scope of changes, or even get a "pass" by tweaking test conditions. Direction, boundaries, and acceptance criteria still need human control.

4.1 Where AI Automation Excels

1. Completes cross-file, rule-clear batch changes

Build migration and platform adaptation pull in a host of related changes—dependency declarations, compile options, macro definitions, test directories, CI scripts. AI can sort these out and continuously implement and verify.

Example: The CMake migration covered 31 build files and supporting scripts; ARM adaptation went from dependency installation, test compilation, process startup to result-file integration.

2. Continues multi-round fixes based on execution feedback

Compile errors, link failures, and test anomalies all give AI clear feedback, allowing it to keep working in an "execute—locate—fix—retest" loop.

Example: In ARM adaptation, AI successively handled missing dependencies, interface bridging, link conflicts, and test framework compatibility, saving the cost of humans repeatedly typing commands and checking results.

3. Reveals hidden assumptions and preexisting problems via regression tests

Changing the build method or running platform surfaces problems that weren't visible in the original environment.

Example: CMake migration's test execution exposed hardcoded paths, test state pollution, and memory management defects;

Example: ARM adaptation also exposed out-of-bounds, uninitialized memory, and use-after-free in tests. AI can keep tracking these failures, and it also runs the full regression and repeated-run verification after fixes.

4.2 Where AI Automation Goes Wrong

1. Tends to adopt depth-first investigation, digging deeper on one assumption

After finding a plausible explanation, AI tends to follow it deeper, adding reasoning, experiments, and code changes without first comparing other possible causes or checking whether the initial assumption holds. It may spend a lot of time analyzing deep mechanisms when the real problem is a compile parameter, dependency path, or test config mismatch. Many investigation steps and detailed explanations don't mean the direction is correct.

2. Tends to attribute problems to base libraries or the OS when evidence is missing

When hitting crashes, memory anomalies, or cross-platform differences, AI may prematurely suspect memory allocators, system libraries, compilers, or the OS, ignoring the project's own call errors, link configs, and test defects.

Example: During ARM adaptation, we had to explicitly ask AI to retract its generalized suspicions about dependency and system libraries, and to add information about existing static libraries and local compilation methods. Base components can be listed as investigation candidates, but they need concrete evidence support; they can't be the default answer for unknown problems.

3. Treats eliminating errors or green tests as the problem being solved

Example: In CMake migration fix records, AI used _exit to skip global destructors to avoid exit crashes, and adjusted assertions after introducing an always-successful auth stand-in; ARM adaptation's CI skipped three test cases that depend on a real KMS. These treatments might be justifiable in a specific test environment, but they change the verification scope, so you can't conclude the original problem is solved or the real dependency chain has been verified.

4. Unnecessarily expands modification scope to solve local problems

AI might change source code outside the task scope, adjust shared build logic, or bypass the current failure by loosening checks.

Example: In ARM adaptation, there were source and format changes that needed reverting, and a global link option that allowed duplicate symbol definitions. The current problem was bypassed, but it could bring compatibility issues or cover up subsequent errors.

4.3 Recommendations for AI Automation

1. Define goals, boundaries, and acceptance criteria before letting AI run continuously

State precisely what may change, what must be preserved, where dependencies come from, and what counts as completion. Within set constraints, AI can autonomously handle builds, tests, and local fixes; anything touching business semantics, public config, or test standards requires human review and decision. Key operations (changing public config, deleting data, releasing to production) always require human confirmation. Permission levels per Section 5. (Corresponds to 4.2 #3, #4)

2. Compare candidate directions before deep verification, set stop conditions for investigation branches

When a failure occurs, first have AI list the main candidates, existing evidence, and minimal verification methods. Prioritize factors directly related to this change and cheap to verify. After each experiment round, reassess direction. If several consecutive attempts yield no new evidence, return to the candidate set; don't keep adding speculation to the same hypothesis. (Corresponds to 4.2 #1)

3. Before suspecting base components, check your own usage first

Prioritize checking compile macros, header and library versions, link order, interface signatures, run directories, and test initialization. Don't attribute to base libraries, compilers, or the OS until you have an independent minimal reproduction, clear version comparison, or direct field evidence. (Corresponds to 4.2 #2)

4. Keep verification of original behavior

Keep the old build as a control; verify old and new architectures, different build modes, and repeated runs. Record root-cause fixes, temporary workarounds, and unverified items separately. (Corresponds to 4.2 #3)

5. Turn the execution process into reproducible, reviewable deliverables

Keep each round's failure phenomenon, modification basis, and retest results. The final deliverable includes code, build steps, environment dependencies, and verification boundaries. Human review should look at not only the code but also the automation script's operational scope and whether the final conclusions exceed actual evidence.

5. Reusable Methodology

The scenarios in Sections 2 to 4 differ, but several principles apply generally:

• Start with low-risk, easily verifiable scenarios: Don't aim for fully automated development. First use AI on tasks whose results can be tested or re-reviewed, then gradually expand scope.
• Every conclusion must be falsifiable: Whether a review finding, a debug hypothesis, or an automated fix, write out trigger conditions ("holds under condition X"), supporting evidence, and how to prove it wrong. Without evidence, mark it as assumption or to-be-verified.
• Tests must be able to turn red: Key verifications must complete the red-green loop: reproduce and turn red reliably before the fix, turn green under the same conditions after the fix, and turn red again when the fix is reverted. Reproductions should go through real entry points, let the system produce the intermediate state itself, and avoid directly mutating internal objects or feeding values that production can never reach.
• Test green doesn't mean problem solved: Changing assertions, introducing stand-ins or mocks, skipping cases, or bypassing cleanup paths all change verification scope and need explanation of the change in coverage. Review why the test passed, and from global invariants, confirm the fix solves a class of problems, not just the trigger case.
• Make domain knowledge explicit, but only the stable parts: Architecture descriptions, key invariants, review checklists, failure cases, and test specs should be written down for AI and maintained alongside the code, so outdated info doesn't mislead AI. See 2.4 for specifics.
• Establish layered authorization: read-only analysis, controlled modification, controlled execution, human decision, each progressively opened up; root-cause determination, fix adoption, and key operations remain with humans.

6. Conclusion

The practices above boil down to three judgments:

First, AI has moved the bottleneck from "writing" to "verification." Code production is cheaper, but each line still needs to be proven correct. How much a team gains from AI depends not on how much code is generated, but on whether verification capability keeps pace with generation speed.

Second, AI's errors are not in the code; they're outside the code. Almost all overturned AI conclusions were paths that work in code but can't be reached in reality: operational flows block them, design decisions deny them, team conventions already discussed them. Filling this gap isn't a matter of a stronger model, but of explicitly written-down and continuously maintained invariants.

Third, fluency is not evidence, consistency is not correctness, green is not resolution. These are the three illusions AI most easily produces. In systems that cannot fail, every conclusion must be falsifiable. A conclusion that can't be falsified, however complete, is just an assumption.

So the conclusion isn't "use AI cautiously" but something more specific: AI expands the search space and accelerates iterative verification; humans own system semantics and final judgment; the boundary between the two is drawn by evidence that can be reproduced in the actual end environment. The clearer this boundary, the larger the scope AI can work on autonomously.

---

Follow the WeChat official account "美团技术团队" (meituantech) or visit https://tech.meituan.com/ for more technical articles.

---

This article is produced by the Meituan Technology Team, with copyright owned by Meituan. Non-commercial reproduction or use for sharing and communication is welcome, with attribution to "Content reproduced from Meituan Technology Team." Unauthorized commercial reproduction or use is prohibited. For any commercial use, please email [email protected] for authorization.