跪拜 Guibai
← All articles
Artificial Intelligence · AI Programming

Stop AI from Copying Legacy Code: Purge Bad Samples First, Then Lock with Types and Tests

By ENCI ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Western engineering teams using AI assistants face the same failure mode: models replicate whatever patterns dominate the repo, including workarounds and dead code. This approach gives a concrete, order-sensitive remediation path: purge the bad examples before tightening the gate, or the gate becomes another ignored rule.

Summary

AI coding models imitate what they see in context, and a codebase full of old APIs, temporary patches, and commented-out dead code becomes a training set for bad patterns. Rules in prompts or READMEs are soft constraints that models can ignore, while type errors and failing tests are hard rejections.

The article lays out a two-step governance order: first lower the density of bad examples by counting and deleting deprecated calls, patch markers, and duplicate blocks; then push constraints into code with enums, parameter objects, return types, and private encapsulation. Add tests that fail on regressions, run three layers of gates in CI, and write error messages that tell the model the replacement, not just a ban.

Order matters. If you enable strict linting before cleaning the repo, you'll face a wall of existing errors and end up adding ignore rules, which turn the gate into a reminder. Clean first so you can set a zero-tolerance threshold, then lock. A runnable Python script in the article scans for outdated calls, patch markers, and duplicate function bodies, exits 1 on any hit, and can be dropped into CI.

Takeaways
— AI copies repo patterns because the training objective is probability, not correctness; context examples are weighted by recency and similarity.
— Four common bad sample shapes: deprecated APIs, temporary patches like FIXME, copy-paste blocks, and commented-out dead code. All are syntactically legal, so they pass review and only break at runtime.
— Prompt rules are soft constraints; compilers and tests are hard rejections. Only the latter reliably stops the model.
— Cleaning order: count first with a scanner, sort by spread speed, wrap old calls into one entry, convert patches to issues or delete, remove dead code, and clean docs and examples too.
— Types seal wrong patterns: enums instead of strings, parameter objects, tightened return types, private members with a single public entry.
— CI gates: static checks, unit tests, and custom rules. Error messages should suggest the replacement, not just forbid the old call.
— Never create a legacy archive directory with exempted checks; it becomes an officially endorsed bad sample.
— Clean before locking. If you lock first, you'll be forced to add ignores that let models keep copying legacy patterns.
— For large legacy debt, use decreasing quotas per directory: zero tolerance for new code, old directories scale down by 0.9x weekly.
— Small or throwaway projects (under 5k lines, 3-person team) only need the health check script; full enforcement isn't worth the cost.
Conclusions

The article's core claim is that AI misbehavior is a sample distribution problem, not a model intelligence problem. That reframes code review: you are curating the curriculum the model reads.

The emphasis on order (purge before gate) is the sharpest point. Most teams jump to linting and then drown in ignore rules, which quietly converts a hard constraint back into a soft one.

The script's duplicate detection by function body fingerprint is deliberately simple, and the article admits its limits. That honesty strengthens the practical advice: use jscpd for real production checks.

The suggestion to delete commented-out code instead of keeping it as a record is counterintuitive to many developers, but it directly addresses the way models weight every token in context as evidence.

Concepts & terms
Soft vs. hard constraints
Soft constraints are instructions in prompts or style guides that change the probability of output but can be ignored. Hard constraints are enforced by tools like type checkers, linters, and tests: the code either compiles and passes or it doesn't, so the model cannot proceed past them.
Bad example density
The proportion of legacy patterns (deprecated calls, patches, duplicates) in a codebase. High density makes it more likely that an AI model copies stale code because the model follows the majority pattern in context instead of consulting external documentation.
Function body fingerprint
A hash of the normalized AST dump of a function's body, used to detect copy-pasted logic across files. Two functions with the same fingerprint are considered duplicates, though it can't catch semantic equivalents written differently.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗