AI Code That Runs Is Still Probably Wrong

When using AI to write code, we often encounter a situation:

• The code format is very complete.
• Variable naming looks very standardized.
• There are no obvious syntax errors.
• Running simple examples also yields results.

But when placed into a real project, problems arise.

This is "looks right, but is actually wrong."

The problem is not necessarily that AI can't write code, but that the code given by AI is usually based on speculation from existing information. If the requirements, project context, or verification process are incomplete, the code may be inconsistent with the real goal.

This article focuses on solving one problem:

After AI generates code, how should we judge whether it can actually be used?

1. Running Doesn't Mean Correct

The correctness of code includes at least four levels:

Check Level	Questions to Confirm
Syntax	Can the code be parsed correctly
Runtime	Will it throw errors during execution
Logic	Does the result conform to business rules
Engineering	Is it safe, maintainable, and won't affect other functions

Many people only check the first two items: the code has no syntax errors and can run normally.

But the real problems often lie in the latter two.

For example, a function that calculates a discount, even if it can return a number normally, does not mean the discount calculation is necessarily correct.

2. Why AI Generates "Looks Right" Code

1. Lack of Business Context

The same field can have different meanings in different projects.

For example, discount might represent:

• 0.2, meaning 20% off.
• 20, meaning a 20% discount.
• 2000, meaning a discount amount of 2000 yuan.

If you only tell AI "write a function to calculate a discount," it cannot determine the real meaning of this field and can only guess based on common writing styles.

2. Using Seemingly Reasonable APIs

AI might generate a function with a reasonable name, or use an already outdated API.

The code looks like it conforms to a certain framework's style, but the current project version might not support it.

Therefore, when seeing AI use unfamiliar methods, you should confirm through project documentation, type definitions, or official documentation.

3. Ignoring Boundary Conditions

AI-generated code usually satisfies normal inputs first.

But real projects also need to consider:

• Null values.
• Negative numbers.
• Maximum and minimum values.
• Decimal precision.
• Error types.
• Duplicate operations.
• Network or database exceptions.

If these requirements are not explicitly raised, AI may not actively handle them completely.

4. Mistaking Example Code for Production Code

To illustrate an idea, AI sometimes omits:

• Parameter validation.
• Permission checks.
• Logging.
• Error handling.
• Timeouts and retries.
• Database transactions.

Example code is suitable for helping us understand the direction, but cannot be directly equated with code that can go live.

3. A Typical Error Case

Suppose the business requirements are:

• Product prices are in yuan.
• discount is passed as an integer percentage.
• discount: 20 means a 20% discount.
• The discounted price cannot be less than 0.
• The price result retains two decimal places.

We ask AI to generate a function:

function calculateFinalPrice(price, discount) {
  return price * (1 - discount);
}

This code looks very concise, but executing the following code:

console.log(calculateFinalPrice(100, 20));

The result is:

-1900

The reason is that the code treated 20 as 20.0, not 20%.

The correct calculation method should first convert the percentage to a decimal:

function calculateFinalPrice(price, discount) {
  if (!Number.isFinite(price) || price < 0) {
    throw new Error('price must be a number greater than or equal to 0');
  }

  if (!Number.isFinite(discount) || discount < 0 || discount > 100) {
    throw new Error('discount must be a number between 0 and 100');
  }

  const finalPrice = price * (1 - discount / 100);

  return Number(finalPrice.toFixed(2));
}

This code still needs to be confirmed according to the project's actual rules, but at least it handles several key issues:

• Clarifies that the unit of discount is a percentage.
• Checks whether the price and discount are valid numbers.
• Prevents the discount from being less than 0 or greater than 100.
• Handles precision for the monetary result.

This example illustrates:

AI didn't necessarily write the syntax wrong, but it may have misunderstood the field's meaning.

4. What to Check First After Code Generation

Step 1: Re-read the Requirements

Don't run the code immediately after receiving it. First, check against the requirements:

• Are the input parameter types correct.
• Are the field units correct.
• Does the return value conform to the agreement.
• Are the success conditions complete.
• How should failures be handled.
• Does it include permission and security requirements.

Pay special attention to fields prone to ambiguity, such as amounts, time, ratios, status values, and IDs.

Step 2: Identify the Assumptions AI Made

You can directly ask AI:

Please list all the assumptions you made when generating this code.

Focus on checking:
1. The type and unit of each parameter
2. How null and outlier values are handled
3. Calculation rules for time, amounts, and ratios
4. Dependent libraries and versions
5. Permission and security premises

Do not modify the code, only list assumptions and issues that may need confirmation.

This step is very useful because hidden assumptions are often the source of errors.

Step 3: Prepare a Minimal Test Set

Prepare at least four types of inputs:

Test Type	Example	Purpose
Normal Value	Price 100, Discount 20	Check the main flow
Boundary Value	Discount 0, 100	Check the minimum and maximum range
Abnormal Value	Null, string, negative number	Check parameter validation
Extreme Value	Very large price and decimals	Check precision and stability

Corresponding to the previous function, you can first write tests:

console.log(calculateFinalPrice(100, 20));
console.log(calculateFinalPrice(100, 0));
console.log(calculateFinalPrice(100, 100));

try {
  calculateFinalPrice(100, 120);
} catch (error) {
  console.log(error.message);
}

The expected results should be:

80
100
0
discount must be a number between 0 and 100

Testing is not to prove AI is definitely correct, but to expose errors as quickly as possible.

Step 4: Check the Project Environment

After moving code from a standalone example into the project, also confirm:

• Are the used libraries already installed.
• Does the import method conform to the current version.
• Does the project use TypeScript type constraints.
• Does the return format conform to existing interfaces.
• Do logging and error handling conform to project standards.
• Is it necessary to supplement unit tests.

If AI uses functions or dependencies that do not exist in the project, you cannot casually install new packages just to make the code run. First confirm whether the project already has similar capabilities.

5. Four Common Errors to Focus on Preventing

1. Logic Correct, Business Wrong

The code runs according to some logic, but does not conform to product rules.

For example:

• Treating calendar days as working days for calculation.
• Understanding "20% off" as "price multiplied by 20%".
• Treating order status 1 as completed, but in the project 1 means pending payment.
• Treating the user's local time as server time.

This type of problem can only be confirmed through requirements, interface documentation, and business examples, not just by looking at syntax.

2. Normal Data Correct, Abnormal Data Wrong

For example, a search function works normally when entering keywords, but throws an exception when entering an empty string, special characters, or an overly long string.

Need to actively test:

• Empty string.
• String with only spaces.
• Special characters.
• Overly long content.
• Parameters of incorrect types.

3. Works Locally, Not in the Project

Common reasons include:

• AI used the wrong framework version.
• The local example depends on uninstalled packages.
• Environment variable names are inconsistent.
• Interface fields differ from project conventions.
• Real asynchronous and permission flows were not considered.

Therefore, after generating code, it must be run back in the real project, not just tested in isolated code snippets.

4. Functionally Correct, but Security Issues Exist

Code that implements a function does not mean it can be used safely.

Need to focus on checking:

• Is SQL directly concatenated.
• Is user input inserted into HTML.
• Are permission checks bypassed.
• Are sensitive fields like passwords and Tokens returned.
• Are detailed server errors returned to the user.
• Are keys hardcoded in the source code.

Security checks should be done separately; do not default to "safe if functional tests pass."

6. Let AI Help You Verify, Not Just Generate

You can use this Prompt:

Please do not directly modify the code below; first help me verify if it meets the requirements.

Requirements:
[Paste complete requirements and business rules]

Code:
[Paste code]

Please output in the following order:
1. What the code currently implements
2. What unconfirmed assumptions the code has made
3. Places inconsistent with the requirements
4. Normal, boundary, and abnormal scenarios that need testing
5. Possible security risks
6. Issues that still cannot be determined

Please mark each issue: High, Medium, or Low priority.
Only after I confirm the issues, then provide a modification plan.

7. Don't Just Look at AI's Explanation

AI's explanation of code can be very fluent, but fluency does not equal accuracy.

When verifying answers, it is recommended to combine the following methods:

• Run minimal examples.
• Check project type definitions.
• Read the official documentation of dependent libraries.
• Search for existing similar writing styles in the project.
• Add normal, boundary, and abnormal tests.
• Use code review tools to check the scope of modifications.
• Involve colleagues for manual review when core logic is involved.

For code related to amounts, permissions, payments, data deletion, and privacy, verification standards should be stricter.

8. AI-Generated Code Acceptance Checklist

Before submitting code, you can use this checklist:

• I can explain this code in my own words.
• Parameter types, units, and value ranges have been confirmed.
• Normal inputs have been tested.
• Boundary inputs have been tested.
• Abnormal inputs have been tested.
• Dependencies and APIs are consistent with the current project version.
• Return results conform to interface agreements.
• No permission and security checks have been omitted.
• No fake data from example code has been brought into the production environment.
• Existing project tests have been run.
• The file scope of this modification has been checked.
• I know why this code is implemented this way, not just that it can run.

If the last item cannot be confirmed, it means this code is not yet suitable for direct submission.

Summary

Common reasons why AI-generated code "looks right, but is actually wrong" include:

• Insufficient business context.
• Ambiguity in parameter and field meanings.
• Using outdated or non-existent APIs.
• Ignoring boundary conditions and exception handling.
• Directly treating example code as production code.

When using AI for programming, don't just ask "Can the code run?", but also continue to confirm:

Does it meet the requirements? Does it cover boundaries? Is it suitable for the current project? Is it safe?

You can remember a simple process:

Read requirements first
  ↓
Confirm assumptions
  ↓
Run minimal examples
  ↓
Test boundaries and exceptions
  ↓
Check project environment and security
  ↓
Then submit code

AI is responsible for increasing coding speed, developers are responsible for verifying whether the results are truly reliable.

The next article will introduce:

"Building Your AI Programming Workflow from Scratch"

---

✍Insist on originality, seeking follows, likes, and favorites