跪拜 Guibai
← Back to the summary

Vague Prompts Cut AI Code Accuracy by 40% — Here Are the 5 Gaps to Close

"The greatest enemy of communication is the illusion that it has been achieved." — William H. Whyte (sociologist, urban planner, Fortune editor, 1950)

When AI writes bad code, people tend to blame the AI first — but the vast majority of the time, the problem lies in the requirements (Spec), not the AI. For the exact same feature, writing a single line like "make me a nice user dashboard" versus writing precise requirements that spell out scope, constraints, and acceptance criteria will produce drastically different results using the same AI. Vague, incomplete, or self-contradictory task descriptions can cause AI code accuracy to plummet by 20% to 40% (data from the academic study When Prompts Go Wrong: Evaluating Code Model Robustness to Ambiguous, Contradictory, and Incomplete Task Descriptions, arXiv:2507.20439) — this is not mysticism, but a conclusion drawn from rigorous scientific experiments. Over the next ten minutes, I will lay out the 7 rules and 1 checklist I have distilled specifically to plug the holes that cause AI to go rogue. You will also understand why writing good requirements is the only "perfect weapon" you can 100% control. AI is like a programmer who "never asks questions": it executes exactly what you write, taking your words literally without understanding your "good intentions."

Core Takeaway — The One Thing This Article Wants You to Know

The single most critical point is: When AI encounters something unclear, it will absolutely never proactively ask for clarification. A senior programmer who spots a gap in the requirements will come to you to confirm; AI silently fills the gap using the highest-probability "template" from its training data — and then perfectly misses your actual intent. The result is code that is "precisely wrong": syntactically flawless, logically coherent, but fundamentally not what you wanted.

The practical implication is simple: You cannot control the upper limit of AI's intelligence, but you can absolutely control the quality of your requirements document. A qualified AI requirement must clarify the five things AI is most prone to guess wildly about:

Remember one core principle: requirements are not about being as long as possible, but about "Signal Redundancy." Communicating the same requirement in different ways (e.g., text description + constraints + examples) is far more effective than writing a whole essay of fluff. If you remember only one sentence from this article, make it this: Before blaming the AI, check which of these five things you left out.

Now that we know the key points, let's look at the principle behind them — why does a single requirement gap inevitably turn into a pile of bad code?

Why Requirement Quality Determines Everything

AI's brain does not have the option of "I don't know, I need to ask." When your task description has gaps, AI does not pause; it directly continues the sentence using the highest-probability words. Worse, a single sensitive word you write can activate a fixed "template" in its brain. For example, if you write "sort and return the top N," AI will happily apply the most classic template it has seen across tens of thousands of training samples — even if that template is completely unsuitable for your current scenario.

This is the mechanism of "hallucination caused by gaps." The code is syntactically impeccable but wildly wrong in business logic, because no one told it what it did not know. In other words, the error is not conjured out of thin air by the AI; it is created by the blanks you left.

The data backs this up. A rigorous controlled experiment showed that vague, incomplete, or contradictory requirements can drag down AI's first-attempt code accuracy by 20% to 40%, with "self-contradictory" requirements being the most damaging (a drop of 40%). It should be noted that this study was conducted on specific evaluation sets (HumanEval and MBPP); real-world development is more complex, but the pattern holds: the moment requirements get a little fuzzy, the code easily becomes "chickens talking to ducks." Various bug analyses throughout software engineering history prove the same point: the majority of bugs in production environments actually originate from unclear requirements at the outset, not from coding errors.

To understand this principle, we don't need to look at dry test data; one famous historical lesson is enough: the "Mars Climate Orbiter" crash. In September 1999, this probe, which had flown for a full nine months, instantly burned up while attempting to enter Mars orbit. The reason is almost laughable: Lockheed Martin, which built the spacecraft, used "imperial" units (pound-force) in its software, while the Jet Propulsion Laboratory (JPL) responsible for navigation assumed "metric" units (newtons) — neither side unified the units when interfacing. The data was transmitted perfectly; sending, receiving, and storage had no errors whatsoever. But over nine months, this tiny "unit misunderstanding" caused the spacecraft's trajectory to deviate more and more. As the accident investigation board stated, it died "extremely precisely and undeniably." And today, the way AI accepts our instructions (tokens) is exactly the same as how that software accepted numbers back then: it eats whatever you feed it, without ever asking what those numbers actually represent.

However, to be fair, we cannot place all the blame on the instructions. The statement "as long as you write good requirements (Spec), AI will do a good job" is not absolute. In some cases, even perfect instructions cannot save the situation.

For example, a real-world comparative test involving senior programmers showed that sometimes, AI actually made these experts work slower — even though they subjectively "felt" faster. In this case, writing more perfect instructions did not rescue the efficiency decline (data from METR research, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, arXiv:2507.09089; test subjects were 16 developers using Cursor Pro + Claude 3.5/3.7 on their own open-source projects; the authors note the results do not represent all software engineering scenarios). Additionally, star AI models that score 84% to 89% on "mock exams" (academic test sets) can see their scores crash to 25% to 34% once placed in a real work environment (e.g., generating complete Java classes from scratch) (data from Beyond Synthetic Benchmarks: Evaluating LLM Performance on Real-World Class-Level Code Generation, arXiv:2510.26130). This is not only because requirements were poorly written, but also because AI lacks global awareness of the entire project or fails to capture context. So, the conclusion is not "AI does a bad job entirely because your requirements are poorly written." Rather, it is: among the many factors influencing AI performance, writing good requirements (Spec) is the single biggest lever you can fully control. Before you complain that "this AI is too dumb, totally unusable," spend a little effort checking your task description against the "five pain points" — this is absolutely the most cost-effective troubleshooting method. It helps you pinpoint where the problem actually lies.

Since "communication gaps" are the root of the problem, the question becomes: what specific methods can we use to bridge this gap?

Core Technique: Seven Golden Rules for Writing Requirements That Make AI Understand Human Speech

For teams, there is good news: these core elements are countable and learnable — this requires no talent, just a bit of discipline and standardization. The following seven rules are a consensus distilled from countless development tools, research papers, and the practical experience of frontline practitioners. They are powerful not because they prescribe some rigid format, but because they precisely fill those communication gaps. Remember, format is just a tool; solving the problem is the ultimate goal.

Rule 1 — Focus on "Outcome," Not "Implementation"

Describe more of what behavior the system should exhibit, rather than teaching it how to write code. Tell the AI what ultimate goal the system should achieve and how success is measured, not what loop statements to use. Stuffing pseudo-code into a requirements document backfires: it biases the AI's thinking, conflating "what to do" with "how to do it." Your role is the "architect" planning the ultimate goal; how to lay the bricks is the "construction crew" (AI)'s job.

Rule 2 — Not Only Say "What to Do," but Also "What Not to Do"

Explicitly pointing out what this change does not involve is just as important as clarifying what it does. Without drawing clear boundaries, AI acts like a contractor without a contract — it will cleverly add features it thinks are "reasonable" but that you don't need at all. A simple sentence like "Do not export attachments, do not modify the database table structure" can save you hours of cleaning up redundant code afterward.

Rule 3 — Write "Testable" Acceptance Criteria

This is the core of writing a good requirements document. Acceptance criteria must be testable, with clear boundaries, black-and-white, and focus on one thing at a time. Vague phrases like "runs smoothly" are absolutely unacceptable. Conversely, "Under 10,000 records, 95% of list API response times must be under 200 milliseconds" is a good criterion because a tester can independently verify whether it passes without asking anyone.

Here is a plain-language standard: If a requirement cannot be proven as passed without asking the product manager, it is still a draft, not a formal requirement. Good acceptance criteria both guide the AI and give you a solid basis to "reject" the code when you find the AI writing nonsense.

Rule 4 — Force Writing Error Flows and Edge Cases

In real-world operation, the smooth "happy path" actually accounts for only a minority of cases, yet most people only write this one scenario in their requirements. If you don't tell the AI what to do when "the list is empty," "no permission," or "limit exceeded," the AI will guess wildly at a set of error handling based on patterns it learned before. You must at least provide: one error flow, one edge case, and one empty data or no-permission scenario. The places where code is most likely to crash after going live are often these three gaps you failed to clarify.

5. Rule 5 — Provide Sufficient Context and Constraints

AI does not understand your codebase. You need to proactively tell it: what tech stack and versions are used, what coding standards apply, which third-party libraries are forbidden, what the output format is, and hard performance limits. If you don't say it, it doesn't mean AI will remain silent — it will start "blind guessing." This is the same principle as missing requirements: where you haven't clarified things, AI will fill in with the most common patterns from its brain, and that pattern is highly unlikely to suit your project.

6. Rule 6 — Give Concrete Examples, Not a Bunch of Rigid Rules

Providing a few typical, diverse "input-output" examples is far more effective than writing a long, tedious list of rules and special cases. Official guides from major AI vendors confirm this: rather than listing a bunch of "exceptions," it's better to directly show it carefully chosen examples. For AI, a picture or an example is worth a thousand words. Providing a real sample of "what correct output looks like" reduces misunderstandings more than writing a whole paragraph of explanation.

7. Rule 7 — Match Complexity and Detail Level

"Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." — Antoine de Saint-Exupéry (author of The Little Prince)

To be honest: regarding "whether requirements should be written short or long," the industry has not reached a conclusion. Anyone who pats their chest and guarantees an absolute standard is fooling you. Proponents of "write short" argue: AI's attention is limited; writing hundreds of lines of requirements just to display a date is a disaster. Proponents of "write detailed" cite research: detailed descriptions filled with constraints, inputs/outputs, and edge cases significantly improve AI accuracy; omit one constraint, and accuracy immediately drops. A smart compromise is: don't fixate on length, but match the level of detail to the difficulty of the task. If you just want to center a button, write it short; if you need to implement an OAuth flow with token refresh, write it in full detail. Most crucially, communicate the same requirement through multiple channels (description, constraints, examples), rather than counting how many lines the requirements document has.

These rules all sound reasonable. Next, let's see what the same feature looks like under a "terrible write-up" versus an "excellent write-up."

Same Feature, Two Radically Different Requirement Descriptions

Let's look at a real task: building an endpoint that exports a user list as a CSV file. This is the version I see most often:

Make an endpoint to export users to CSV, remember to make it nice to use.

Imagine if an AI model (or programmer) that never proactively asks for details sees this sentence, what will it do? What CSV format? What delimiter? What encoding? — Not written. Which columns to export? All fields or only public info? — Unknown. What if the user list is empty? — Undefined.

Who has permission to call this endpoint? — Unresolved. If exporting 100,000 records at once, will the system freeze? — Entirely up to luck. Faced with these blanks, AI can only fill them by "wild guessing" — and probabilistically speaking, whenever there's guessing, there will always be times it guesses your intent wrong.

Now, let's look at the same task after plugging all these "holes":

Expected Outcome: Download a CSV file containing the list of "active users" via the GET /users/export endpoint.

Scope: Active users only. Export columns: id, email, created_at.
Non-goals: Do not export deactivated accounts, do not export billing data, do not modify any data.

Constraints: Use UTF-8 encoding, comma-separated, first row as header; use streaming response for datasets exceeding 5000 records; admin permission only.

Acceptance Criteria:
- If there are 3 active users, return 3 data rows + 1 header row.
- If there are no users, return only the header row with status code 200.
- Non-admin calls return a 403 error and do not download any CSV content.
- Under 50,000 records, response time is within 2 seconds (using streaming).

Example Output:
id,email,created_at
42,[email protected],2026-01-12

The second version is not actually long-winded — it simply provides precise instructions in the places where the first version "played deaf and dumb."

Rule 1 replaced the vague "make it nice to use" with a clear "Expected Outcome"; Rule 2 cut out irrelevant attachment and billing export needs; Rule 3 broke down "runs smoothly" into four verifiable specific conditions; Rule 4 standardized the two special cases of "empty data" and "no permission"; Rule 5 locked down the encoding format, delimiter, and streaming threshold.

This precisely proves our initial point: What determines the quality of the final result is not the word count, but whether you have plugged these five "holes."

Even a requirements document (Spec) written this way is rarely flawless on the first draft. The key is learning how to refine and iterate it step by step.

How to Refine a Requirements Document (Spec) Step by Step

A requirements document is not something you finish writing and then shelve forever. It is "living," growing within a cycle: you write tests, write code, see where things break, then fill in the parts of the requirements document that weren't clear, and repeat the process.

Testing not only catches bugs in the code but also shines a light on gaps in the requirements document. A failing test often signals: "Hey, the rules here weren't fully written before!" rather than just a coding error. In this process, a clear, step-by-step workflow can have an immediate and dramatic effect. In a study by AlphaCodium (Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering, arXiv:2401.08500), researchers changed the instructions given to GPT-4 from a "one-shot package dump" to "step-by-step guidance," and GPT-4's success rate on high-difficulty programming competition problems jumped directly from 19% to 44% (this was on the CodeContests competition dataset. Although competition problems differ from our daily practical development work, this trend aligns with other observations about "iterative methods," all demonstrating the power of step-by-step iteration).

This cycle also gives rise to a simple but golden rule: When code and the requirements document conflict, the document always prevails. The requirements document is the "contract"; the code is merely the "specific plan for executing the contract." A test failure doesn't necessarily mean the code is wrong; sometimes it is actually sounding the first alarm: your document has started to drift from reality. This is the seed of "Specification Rot" — where the document and actual code gradually go their separate ways — but that is a story for another day.

Everything has two sides. The flip side of mastering the discipline is knowing when not to write a requirements document (Spec). For example, fixing a small bug, building an exploratory prototype, or a task that can be verified with a single unit test — in these cases, writing a full document is purely a waste of time, not worth the effort. Remember one principle: Invest effort proportional to the risk.

If you can verify the result with a simple test, the blast radius is small, and you can proceed freely. But whenever permissions, payments, or data integrity are involved, you must write the document properly no matter what. Additionally, there is an "iteration trap" here: AI assistants (Agents) sometimes exploit loopholes, marking a task as "completed" even without writing tests — the so-called "document looks fine, code isn't actually finished" scenario. Therefore, the final gatekeeper must be a human, not the AI.

The entire workflow needs a "quality checkpoint." This is where the "checklist" comes into play — use it for a final review before handing the document to the AI assistant.

Requirements Document (Spec) Validation Checklist

Before sending the document to the AI assistant, run through this short checklist first. This is not bureaucracy, but a "rejection mechanism" (reject and rewrite if it fails), and the cheapest way to cultivate good documentation habits in a team. You can directly copy the version below:

That checklist is your best training material. We need to treat requirements documents (Specs) and prompts the same way we treat code: version-control them, diff changes, and run regression tests before merging and releasing. Reviewing requirements like we do Code Review should become a daily habit.

Circulate examples of excellent and terrible requirements (like those in the previous section) within the team as reference standards. Here is a great trick borrowed from high-security-requirement industries: engineers at Amazon AWS spent only two to three weeks learning how to write rigorous system specifications and immediately caught real-world bugs. The secret was simply packaging those "scary" rules as ordinary, practical daily norms. (This references a 2015 ACM Communications study; although the original text refers to TLA+, a formal model-checking language — which differs slightly from writing natural language requirements for AI — the principle is the same). What often hinders us from writing good requirements is psychological resistance, not technical difficulty.

This is the secret that makes this skill so valuable. Writing a good requirement demands rigorous logical thinking on par with writing good code — precisely because it's not easy, it is exceptionally valuable.

Summary

The entire article boils down to one sentence: The lever you can truly control is "requirements," not the "AI model." The model will not proactively ask you what preconditions you left out — it will only fill in the blanks using patterns from its training data. Therefore, a task description riddled with gaps will systematically mislead the AI, and even if the resulting code looks fine, the outcome is wrong.

Here are the four most valuable takeaways from this article: