100% Test Coverage Is the Missing Evaluation Function for AI Code

Synced to my personal site: Why I now chase 100% test coverage in the AI era?

During a recent company hackathon and in my personal project, I made a seemingly paranoid decision: force the test coverage metric to 100%.

We wrote this requirement into the CI gate, track it in real time with Codecov, and display a badge in the README.

• 3D Web project Spider: https://github.com/minorcell/spider
• Hackathon work Open Anywhere: https://github.com/open-anywhere/open-anywhere

In traditional software engineering, chasing 100% coverage is often seen as formalism or too expensive at the margin. But in the new era of AI coding, that logic has been turned upside down. Here's my recent practice and thinking.

Tests are the "evaluation function" for AI coding

Our development model has fundamentally changed: developers increasingly step back into the roles of product designer and architecture decomposer.

Hand a well-structured proposal and a reasonable architecture decomposition document directly to an AI agent, and it can run nonstop for hours, delivering a feature-complete product.

But large models are trained using an evaluation function (Evaluation Function / Reward Model) to judge whether each iteration is good or bad, so they gradually converge on an optimal solution. So in the step where an AI agent writes code for us, what is its "evaluation function"?

Tests are definitely an important part of that evaluation function.

Without tests, AI generation is like an anchorless drift; and 100% test coverage gives AI a tightly calibrated, deterministic feedback mechanism.

In recent projects, we even introduced a "red team / blue team" mechanism similar to what's used in cybersecurity: the main agent dispatches independent sub-agents that specifically play attack and defense roles to write test cases, digging out hidden edge cases well before code merge.

The practical result was astonishing: at the program-logic level, there were almost zero bugs. The few subsequent fix commits were due to adjustments on product decisions, not code quality defects.

Unexpected bonus: 100% coverage forces AI to eliminate "defensive redundancy"

A common complaint when using AI to write code collaboratively is that AI-generated code is bloated. For a simple logic, it habitually nests several layers of defensive checks (null checks, boundary fallbacks, and so on).

But when we hard-set test coverage at 100%, something interesting happened: AI started actively simplifying its own code logic.

Because strict test frameworks (such as Vitest Code Coverage, Go Code Coverage) require that every branch and every conditional judgment be genuinely hit by a test case. If AI writes five layers of defensive checks, it must write test cases for all five extremely rare branches.

A concrete TypeScript comparison example:

Scenario: write a simple user ID formatting function.

AI's instinct is "safe first," so it habitually writes a lot of code that passes type checks but whose branches may never actually execute.

// formatUserId.ts
interface UserInfo {
  id?: string | number;
  prefix?: string;
}

export function formatUserId(user: UserInfo | null | undefined): string {
  // defensive check 1: input is null/undefined
  if (!user) {
    return 'GUEST_000';
  }

  // defensive check 2: id missing or empty
  if (user.id === null || user.id === undefined || user.id === '') {
    return 'UNKNOWN_000';
  }

  // defensive check 3: handle number id
  let finalId: string;
  if (typeof user.id === 'number') {
    finalId = user.id.toFixed(0); // ensure no decimals
  } else {
    // defensive check 4: fallback to string
    finalId = String(user.id);
  }

  // handle prefix logic
  const prefix = user.prefix ? user.prefix.trim() : 'USER';
  return `${prefix}_${finalId}`;
}

To get this code to 100% Branch Coverage, AI (or a human) must write 6–7 test cases covering:

• Input is null
• Input is {}
• id is ''
• id is a number
• id is a string
• prefix exists, is empty, or is all spaces.

The "simplified" code AI generates under 100% coverage pressure:

When AI realizes the cost of writing tests for those defensive branches is higher than simplifying the logic itself, it tends to use modern syntax and sensible upstream type constraints to simplify.

// formatUserId.ts (simplified)
interface UserInfo {
  // reasonable upstream type constraints, avoiding downstream null checks
  id: string | number;
  prefix?: string;
}

export function formatUserId(user: UserInfo): string {
  // leverage nullish coalescing and optional chaining to simplify branches
  const prefix = user.prefix?.trim() || 'USER';
  
  // leverage String() constructor to simplify type conversion checks
  const finalId = String(user.id);

  return `${prefix}_${finalId}`;
}

To meet the 100% coverage requirement, AI, in self-correcting, is forced to learn to write clean, direct code without redundant defensive wrappers. This "pushing" mechanism makes the generated code visibly more elegant.

From unit tests to benchmarks: reaching 120fps in a 3D web app

Beyond regular unit tests, integration tests, and E2E tests, we also force-added benchmark performance tests on critical paths.

Take our 3D browser rendering project Spider as an example: on the web, 3D scene rendering is extremely sensitive to performance bottlenecks. By embedding benchmark results as a metric in the feedback loop, we let AI keep iterating on and optimizing the code based on performance data.

In the end, this 3D web project hit a stable 120fps in our test environment, and even laptops from four or five years ago run it smoothly.

Try it here: 蛛小八

This closed loop of "test/performance baseline -> data feedback -> AI-targeted refactoring" makes what used to be complex performance tuning quantifiable and automatically iterable.

Conclusion

In the era of human-written code, chasing 100% coverage was limited by human effort and time costs; but in the era of AI agents, the marginal cost of writing tests and refactoring code has been squeezed to the limit.

Tests are no longer a burden on development; they are the reins that control AI's energetic gallop.

If you're letting AI take on large-scale code delivery, try giving it a 100% test coverage "evaluation function" and see what surprises it brings.

• Project source:
• Spider - 3D web browser rendering
• Open Anywhere - hackathon work