Five Checks Before Merging AI-Generated Code: From Semantic Drift to Operational Readiness

Last month I added data validation to a backend service. The requirement wasn't complex: runtime validation for request parameters on a few core interfaces, without affecting existing callers. I packed the requirement into a structured description and handed it to the coding assistant. A few minutes later, six files were changed, with a few unit tests thrown in. Type checks passed, tests were green, and a row of checkmarks filled the terminal.

I was pretty proud at the time. The next day, during integration testing, everything fell apart: null and undefined handling didn't match the backend contract, several edge values passed straight through; the new validation module got entangled in a circular dependency that didn't break the build but caused occasional runtime initialization errors; and one test was asserting the wrong behavior — effectively turning the bug into a fixed expected outcome.

After that, I changed my approach. One sentence sums it up: The quality of your requirement description determines the starting point; the quality of your acceptance determines the finish line. No matter how fast the generation is, whether you can confidently ship it depends on how you review.

First look: Don't rush to the code; align on the requirements first

When you get AI code, a lot of people immediately admire how nice the implementation looks. My order is reversed: I start by checking the business requirements line by line against what it did. AI's biggest pitfall isn't syntax errors — it's semantic drift — its understanding of a word is not exactly yours, but it writes with great confidence.

A real example. The requirement was "spend 100, get 10 off; under 100, no discount." The AI's code was return price * 0.9. At 99 yuan it should not discount, but it did; at 200 yuan it should subtract 10, but it subtracted 20. The code runs, the syntax is pretty, and it's not doing what you asked.

So in the first pass, I take interface signatures, method parameters, return values, and match them one by one against the business rules. If something doesn't align, I send it back for a rewrite. That saves all the later time.

Second look: Read the diff before running any commands

The tests were written by AI, and the code was modified by AI — neither counts as original evidence. The diff does.

git diff --stat

First, look at the scope of changes;

git diff --check

This catches trailing whitespace and conflict markers — AI-generated code leaves these tails more often than you'd think.

Once the scope looks reasonable, I focus on four types of changes:

• Conditionals (a hotbed for semantic drift)
• New imports (can bring in circular dependencies)
• Deleted code (might be defensive logic added by someone else after a painful incident)
• Test files (more on those later)

For dense changes, I compare word-by-word with --word-diff; it's much clearer than line-by-line. Once, it "conveniently" deleted a block of retry logic that looked redundant — but that block had been added after a production timeout incident six months earlier. Since then, I confirm every deleted line.

Third look: Focus on boundaries and security — AI pretends to be fine here

AI writes the happy path fluidly, but at the boundaries it starts to fudge. Empty parameters, array bounds, division by zero, timeouts, concurrent conflicts, third-party API hiccups — it doesn't proactively think about these. The null bug was a classic: the model treated null and undefined as roughly the same, wrote a strict equality check, and null passed through while the type checker stayed silent.

Security is the same; it's not a security expert. SQL string concatenation, hardcoded secrets, logging tokens, user input without validation or escaping — these pass function-level checking but blow up after deployment. It also enthusiastically recommends dependencies you've never seen; check both the source and the version. Don't let an unknown package sneak into production. For modules involving money, user data, or permissions, I read this section line by line, and I don't read for speed.

There's also a class of problems that don't show up locally: N+1 queries in loops, loading entire tables without pagination, lock granularity left to guesswork, and thread pool parameters set arbitrarily. With a hundred rows locally, everything is silky smooth; once production data volume grows, the connection pool gets maxed out first. So for every piece of code, I ask: at production scale, what would it do?

Step four: I don't trust AI's all-green tests

AI-written tests have a subtle flaw: it writes the implementation first, then reverse-engineers the tests from that implementation. It asserts that "the code works the way it was written," which is one step removed from "the code works the way the requirements demand." Such tests are useless even when green; they just give you a false sense of validation.

So I design my own acceptance cases, independent of its tests: a set for normal values, boundary values, empty values, and exception values, listed as input and expected output. I also go further with reverse validation — deliberately tamper with a returned field to see if the test turns red. If a fake case can't turn red, throw it out. After unit tests are complete, integration tests run as normal; the relationship between modules has to be settled separately.

Step five: Say the ugly things before deployment

Whether naming is clear, whether a method has a single responsibility, whether comments are sufficient — these determine whether another team member will have a smooth path or a torture six months later. AI sometimes picks odd names like processData or handleStuff; they work now, but six months later nobody knows what data they handle. Structurally, it tends to over-engineer — one function would suffice, and it wraps three layers of abstraction. AI can write code, but maintenance is ultimately on humans.

Before deployment, also confirm: are logs in place, monitoring metrics in place, alerts in place, and if a failure occurs, how many minutes to roll back, and can data be reconciled after rollback? It won't design these for you. If you don't think ahead, you'll be crawling at 3 a.m. in a panic.

Finally, keep a ledger

The pitfalls I hit in acceptance, I jot down in a document and glance at it before writing every requirement description. After the null incident, my description gained a line: "distinguish between null and undefined." After the circular dependency incident, it gained "confirm the dependency direction before adding a new import." Six months later, the same problems show up noticeably less often during acceptance — prompt and acceptance grow together; the former avoids mistakes, the latter catches the rest.

AI is a capable but irresponsible colleague. The moment code is delivered, its job ends; for every line after that, the person signing off is you. That night, after fixing the null bug, I added entry #27 to the document. What the next entry will be, I don't know yet.