跪拜 Guibai
← All articles
AI Programming · OpenAI · AIGC

AI Code That Runs Is Still Probably Wrong

By 全栈弄潮儿 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

AI coding tools lower the barrier to generating plausible-looking code, which means the real bottleneck shifts from writing speed to verification discipline. A developer who treats a clean run as proof of correctness ships business-logic bugs, security holes, and environment mismatches that compound in production.

Summary

AI code frequently looks correct: clean formatting, sensible variable names, no parse errors, and simple examples that run. The failure mode is not syntax but silent mismatches with real requirements — a discount function that treats a percentage as a raw multiplier, an API that doesn't exist in the current framework version, or missing guards for nulls, negatives, and extreme values. These gaps stem from AI's reliance on statistical patterns without access to the project's specific business rules, field semantics, or environment constraints.

A practical verification pipeline starts by re-reading the original requirements and explicitly asking the model to list every assumption it made — parameter units, dependency versions, error-handling posture. Then a minimal test set covering normal, boundary, abnormal, and extreme inputs exposes logic flaws fast. After that, the code must be dropped into the real project to catch import mismatches, missing packages, and interface contract violations. Security checks — SQL concatenation, hardcoded secrets, bypassed auth — run as a separate pass, never inferred from functional tests.

The article supplies a concrete acceptance checklist and a prompt that turns the AI from generator into reviewer, outputting prioritized gaps before any edits are made. The core rule: if you can't explain why the code is implemented that way, it's not ready to commit.

Takeaways
Code correctness spans four layers — syntax, runtime, logic, and engineering — and most AI verification stops after the first two.
AI guesses field semantics from common patterns; a `discount` field might mean a percentage, a decimal fraction, or an absolute amount, and the model picks one without asking.
Seemingly reasonable APIs in generated code are often outdated or absent from the current project's actual dependency versions.
Boundary conditions (null, zero, negative, max/min, precision, network failures) are rarely handled unless explicitly demanded in the prompt.
Example code from AI routinely omits validation, auth checks, logging, error handling, timeouts, retries, and transactions — it illustrates intent, not production readiness.
A four-step verification sequence — re-read requirements, extract AI's hidden assumptions, run a minimal four-category test set, and validate against the real project environment — catches most silent failures.
Four recurring error patterns: logic-correct but business-wrong, normal-data-correct but abnormal-data-wrong, works-locally but fails-in-project, and functionally-correct but insecure.
A verification prompt that asks the model to list assumptions, inconsistencies, test scenarios, and security risks — without modifying code — turns generation into a review step.
Fluency of AI's code explanation does not signal accuracy; cross-check with type definitions, official docs, existing project patterns, and peer review for core logic.
A 12-item acceptance checklist ends with a single gate: the developer must be able to explain why the code is implemented that way, not just that it runs.
Conclusions

The phrase 'looks right, but is actually wrong' names a failure mode specific to AI-generated code: syntactic and stylistic plausibility masks deep semantic drift from business requirements.

AI's default behavior is to fill gaps with statistically common answers, which makes unstated assumptions the primary source of bugs — extracting those assumptions explicitly is a high-leverage debugging tactic.

The discount-function example is a compact case study in how a single missing unit conversion (percentage to decimal) produces a result that is numerically valid but commercially absurd (-1900).

The checklist's final item — 'I know why this code is implemented this way' — reframes AI-assisted development as a comprehension gate, not a speed gate, which inverts the usual productivity narrative.

Treating AI as a reviewer before treating it as an editor (via the structured verification prompt) changes the human's role from code-accepter to requirement-enforcer, reducing the risk of rubber-stamping plausible output.

Concepts & terms
Four-layer code correctness
A framework that separates code validation into syntax (can it parse?), runtime (does it execute without errors?), logic (do results match business rules?), and engineering (is it safe, maintainable, and compatible with the project?). Most AI-code checks stop at layer two.
Hidden-assumption extraction
A verification technique where the developer asks the AI to list every assumption it made about parameter types, units, dependency versions, error handling, and security context — without modifying the code — to surface the gaps that cause 'looks right, but is actually wrong' failures.
Four-category minimal test set
A testing pattern for AI-generated functions: normal values (happy path), boundary values (min/max of valid range), abnormal values (null, wrong type, negative where disallowed), and extreme values (very large numbers, high precision) — designed to expose errors quickly rather than prove correctness.
From the discussion
Featured comments
咬代码的兽

I feel this deeply. The biggest pitfall of AI code is that its 'language intuition' is too good but it lacks execution feedback — it looks logically closed, but when you run it, all the boundary conditions collapse. My approach is to always pair two checks after generation: boundary-value unit tests + having the AI retell the key logic itself. If the retelling is vague, there's almost certainly a problem. Also, distilling the pitfalls you've stepped on into a checklist is very worthwhile. I usually browse the AI programming category on AI345 and can find plenty of similar practical tools — searching by scenario is much more efficient than blindly searching on my own.

See top comments, translated →
Source: juejin.cn ↗ Google Translate ↗ Backup ↗