Requirements Entropy, Not Token Count, Is the Real AI Coding Cost
In the AI Era, Information Transfer Matters More Than Writing Code
Let me first list two problems I encountered in development:
- Last month, during a requirements review, the product put a twenty-plus-page PRD on the screen. After the meeting, I directly asked AI to split tasks based on the document. The output looked plausible, but when it ran, everything was off. Later I questioned those vague parts one by one, and added more than a dozen "everyone knows" implicit conditions before AI could run it through.
- I was following TDD (write tests first then implement, Red→Green→Refactor). It was accurate but not efficient. Many companies now use locally deployed local models, and their performance and speed are greatly reduced. Development is slow, PM pushes for launch, so this model is not suitable for urgent needs. When facing urgent needs, I need to express accurately to LLM, let it understand my requirements, give an accurate execution plan, and land the code quickly.
After these two events, I became more convinced: in the AI era, the most expensive thing is not writing code, but getting to the point where you don't have to guess. Let's use information theory and the Cynefin framework to break down "making it clear".
1. Quantify: What exactly is information transfer?
Shannon Entropy gives us a measure of a system's "uncertainty":
H(X) = -∑ p(x_i) log₂ p(x_i)
- X: a discrete random variable, such as the possible states of a requirement or the defect distribution of a piece of code;
- p(x_i): the probability of the i-th state, with ∑ p(x_i) = 1;
- H(X): in bits, representing the average amount of information needed to fully describe this random variable.
Intuitive understanding of the two extreme cases:
- When the outcome is certain (p=1), H(X)=0, no message needs to be transmitted;
- When all n outcomes are equally likely (p=1/n), H(X)=log₂ n reaches its maximum, uncertainty is highest, and the most information must be conveyed.
2. Entropy and Cynefin: The Four Domains Are Four Kinds of Uncertainty
We need to correct a common misconception: knowledge itself does not produce entropy; entropy describes how much uncertainty the receiver still has about the thing. The same business logic might have H≈0 in the eyes of a veteran employee, but in the eyes of a new hire or the same AI, it might approach log₂ n.
So documents and processes actually lower the receiver's conditional entropy H(X|D) — they reduce uncertainty on that side, the world itself doesn't change. Tacit knowledge is hard to encode precisely because it isn't written into D, so H(X|D) stays high.
This maps perfectly to Cynefin's four domains, from highest to lowest uncertainty:
| Mode | Receiver's Entropy | Meaning |
|---|---|---|
| Chaotic | Highest | Cause and effect are only clear after the fact; any prediction before is no better than random guessing, H approaches log₂ n |
| Complex | High | Must rely on external knowledge (RAG, human decisions) to supplement before entropy can be reduced |
| Complicated | Medium | Knowledge is sufficient, chain reasoning can bring entropy down step by step |
| Simple | Low | Categorize and respond, H ≈ 0 |
In AI workflows, one core value of LLMs is entropy reduction: transforming high-entropy black boxes in Chaotic and Complex modes into low-entropy sub-scenarios via sensing, analysis, and RAG. Those experiences that cannot be spoken are gradually turned from tacit to explicit through review, exception agents, and performance agent feedback loops. Information transfer matters because it continuously lowers uncertainty in collaboration and cognition.
3. Tacit Knowledge: Why Experience Cannot Be Transferred
What is the experience in your head? Is it something you can write as a tutorial, or only something you can get by hands-on?
The latter covers most of it. For example, two APIs achieve the same function but have different internals, and one is more suitable in a certain scenario; or the same API behaves completely differently in test vs. production. These judgments are never written in documents and cannot be written — they grow from repeated failures. This is tacit knowledge, which exists only in your brain; words and language can only transmit the shape, not the core.
I'm a basketball lover and could never hit three-pointers. I watched many tutorials, followed them, and still missed. Later a coach said: tutorials are just theory; the power and motion have to be felt personally, you cannot just be told. That part that "cannot be taught by words" is tacit knowledge, and it's the ultimate expression of capability.
Applied to AI work, it's the same: problems that tutorials and docs can't solve can only be tackled by experience and intuition. Experience is not read from books; it is tried, summarized (the small model from the previous article works by 'guessing').
The question then: since we can't transfer it, how do we verify it was correctly understood? In the AI era, the answer is review and deliberate upfront comprehension. Let a person externalize implicit judgments once, then let AI restate and execute, and infer from the result whether it truly absorbed the intent.
Why Developers Can't Test Their Own Bugs, But Testers Can
Two reasons.
First, inertial thinking. The code is your own; your brain has run that path dozens of times, so when reading the code your eyes automatically skip parts that don't match expectations.
Second, the type of knowledge differs. The developer works from the PRD and implements explicit knowledge. Testers, besides checking explicit functionality, also run exception handling, smoke tests, boundary values — these procedures are invisible and unspoken. What to test and how to test come from a set of intact "where problems tend to occur" experience.
In the AI pair-programming mode, this can be engineered: build a performance Agent that specifically tests performance, and an exception Agent that specifically tests exception handling. These two agents essentially encode tacit knowledge into executable checklists — test results flow back to the developer, completing the transfer of tacit knowledge.
4. Requirement Transformation: Four Domains, Four Processing Chains
Agents like Claude Code follow a fixed chain: information perception → information analysis → information decision → information execution → information feedback. The difference is which domain each step falls into and how it's handled.
Cynefin has four domains plus a central "Disorder" zone for things you can't yet classify.
Simple: Categorization is Enough
Example: Archive a document with a known format.
Chain: sense → categorize → respond.
Knowledge is sufficient and clear, the rest is execution. Tasks like these can be assigned to rules or a single LLM call; don't overthink.
Complicated: Break It Apart and Analyze
Example: Write a function that sorts by a given rule.
Chain: sense → analyze → respond.
Knowledge is sufficient and immediately available, but it cannot be done in one step — it requires chain reasoning: first sense what to change, then analyze how to change it, then make the change. This step looks most like CoT (Chain of Thought): the longer the chain, the easier it is to go off track, so validation is usually needed.
Complex: Probe Then Converge
Example: Requirement analysis, requirement decomposition, plan mode.
Chain: probe → sense → respond.
Knowledge is insufficient, so external knowledge must be brought in — external here includes not only MCP but also human decisions and the tacit-experience probing mentioned earlier. After supplementing, analyze, generate a TODO list, then execute and see feedback.
Chaotic: Act First to Stop the Bleeding
Example: An intermittent bug nobody can locate.
Chain: act → sense → respond.
Cause and effect are unknown, so the only way is to try something and observe the reaction, working backwards from the reaction to the cause. Once the cause is found, break it into sub-requirements and let those fall back into Simple, Complicated, or Complex — that's the only exit from the Chaotic domain.
5. How LLMs Participate in Mode Transformation
The same model performs vastly differently in different domains. In Simple mode it's fast and accurate; in Complex mode it needs multiple rounds of retrieval and questioning, costing several times more time and tokens; as for Chaotic mode, I basically don't hand it to the model — it doesn't know cause and effect, and it's particularly good at giving a plausible-sounding answer. Such a guess is more dangerous than no answer.
So the real work is not "let LLM solve chaotic problems" but to have a human first drag it out of chaos, breaking it into Simple or Complicated sub-problems, then hand those to the model.
Decomposition: From Black Box to White Box
Problems stuck in Chaotic or Complex mode are usually because they are not clear. The move is to split scenarios into a bunch of as-independent-as-possible sub-scenarios.
The trouble is that sub-scenarios often can't be split easily either, because they are black boxes to us as well. The approach is to use LLM to identify black boxes, open them, then combine the white-box information we have, using CoT chain logic to reduce the complex problem step by step until it lands in Simple mode.
This process is rarely done in one pass; most of the time it's a loop: split, can't see clearly, split again. What it truly improves is the speed of "information perception → information analysis" and the speed of verifying whether an understanding is correct.
6. Time Dimension: Tasks Have a Lifecycle
There's an easily overlooked decomposition clue — time.
Any requirement has a lifecycle: when triggered, when context is loaded, when executed, when unloaded. Slicing along the time axis naturally unfolds complexity, and LLMs are especially sensitive to time signals.
Task Lifecycle (UML State Machine)
A requirement is not a one-time input; it's an entity with states, stages, and trigger timings. Drawing it as a state machine makes complexity unfold along time: each state only needs to consider "what stage am I in now, and what event triggers the next step," without having to understand all details at once.
stateDiagram-v2
[*] --> 待触发 : 需求登记
待触发 --> 装载中 : 定时器 / 事件 / API 触发
装载中 --> 初始化 : 配置 + 上下文注入完成
初始化 --> 执行中 : 资源就绪
执行中 --> 校验中 : 产出结构化结果
校验中 --> 执行中 : 校验不通过(CoT 重规划)
校验中 --> 完成 : 校验通过
执行中 --> 异常超时 : 超时 / 异常抛出
异常超时 --> 执行中 : 退避重试 backoff
异常超时 --> 失败 : 重试耗尽
失败 --> 卸载 : 补偿 / 降级
完成 --> 卸载 : 归档 / 回写知识库
卸载 --> [*] : 释放资源 / 清理上下文 / 留快照
Key Time Anchors
| Time Anchor | State Transition | Engineering Meaning | Who Should Handle It |
|---|---|---|---|
| When triggered | 待触发 → 装载中 | Four trigger sources (timer/event/API/manual) determine SLA | Scheduler + LLM for trigger condition judgment |
| When loaded | 装载中 → 初始化 | Which context, tools, permissions to inject is the biggest Token cost first | Context engineering / Skill / MCP |
| When executed | 初始化 → 执行中 | Entering LLM inference main battlefield, both latency and Token are high | LLM + tool calls |
| When validated | 执行中 → 校验中 | Self-check / test / performance agent intervene to form feedback loop | Exception agent, performance agent |
| When retried | 执行中 ⇄ 异常超时 | Backoff retry to avoid blind repeated consumption in chaotic mode | LLM judgment + retry strategy |
| When unloaded | 完成 / 失败 → 卸载 | Release resources, clear context, keep minimal snapshot; task boundary equals context boundary | Lifecycle management / memory consolidation |
Why LLMs Can Recognize the Time Dimension
- Text natively carries time signals: "every day at 8", "after login", "after successful payment", "timeout after 3 seconds", "before task ends" — these words are triggers in UML, no extra annotation needed.
- Can be extracted as directed state transitions: as long as the LLM can answer "which state must follow the previous state", the state machine is extracted; the more deterministic the description (closer to Simple), the more accurate the extraction.
- Complexity is strongly correlated with time scale:
ms-level (initialization, tool calls) is deterministic;s-level (inference generation) is the cost main battlefield;min~h-level (retry, long tasks) requires memory;d-level (waiting on requirements, experience consolidation) belongs to tacit knowledge. Concentrating cost on high-entropy time segments is the first principle of Token governance. - State equals context boundary: clarifying "when to unload" equals clarifying where the context should be truncated — this is isomorphic to SubAgents' "destroy on task completion" policy. The time dimension is the theoretical basis for Agent lifecycle management.
Relation to Other Two Dimensions
The time dimension answers "when to do", the other two constrain "what to do":
- Structure dimension (the code architecture in the next section): task composition and dependencies determine split boundaries;
- Scenario dimension (the "scenario decomposition" earlier): which Cynefin domain the task falls in determines whether to give it to LLM or human.
Only when the three intersect can a chaotic requirement be cut into independently executable, independently unloadable sub-tasks.
7. Back to Daily Development: How to Split Tasks
What I discussed earlier was the transformation from Complicated to Simple mode, which is theoretical. Now let's switch to the two daily entry points: product documents and existing code structure.
Product Documents as Entry
In the AI era, documents must be clearer and more detailed than traditional ones.
PRD is the first entry for task decomposition and also the biggest gap. Traditional PRDs are written for people with context: readers are teammates from the same team — PM, front-end, back-end, testing — sharing business background, historical baggage, and verbal consensus, so the document can leave many blanks: "handle per the old logic", "refer to page A", "cover exceptional cases as needed". These blanks are filled by humans' experience.
AI does not have this shared context. It only has the information you write. What's not in the document, the model can only guess by probability; the result reads plausible but is unusable. To phrase it in earlier terms: traditional documents shift the cost of entropy reduction to the reader side (readers fill gaps with shared background), while in the AI era you must do the reduction on the author side (write it into the document).
The pipeline from document to task decomposition roughly looks like:
flowchart LR
A[Read doc<br/>mark uncertain points] --> B[Structurally extract<br/>entities/states/rules]
B --> C[Fill black boxes<br/>query each undefined quantity]
C --> D[Align<br/>with existing code/interfaces]
D --> E[Produce tasks<br/>one suite per rule]
E --> F{Rule testable?}
F -- no --> C
F -- yes --> G[Enter Simple mode<br/>directly executable]
Corresponding concrete steps:
- First mark uncertain points, don't rush to split tasks: read through, highlight all "大概/similar/reference/cover" phrases; these are ambiguity markers.
- Structural extraction: break the document into four atomic types — Entity, State, Rule, Action. A sentence that can't be decomposed means you haven't thought clearly.
- Probe black boxes: every undefined quantity is a chance. This is where Complicated-mode eats human time, and where LLM can best assist — it can expand a vague problem into 10 specific questions to confirm.
- Align with existing code: every rule in the document must map to an existing data structure or interface. If it doesn't map, either the doc is wrong or the code is missing; both must be exposed before starting.
- One rule, one task: decompose at a granularity of "one testable rule + one set of cases", not by page. Pages are for humans; rules are for execution.
- Testability self-check: if a task can't produce an assertion, it's not a task, it's a wish. Send it back to step 3.
In the past, PRDs faced humans; now they face AI, so you must write the story completely. I've always believed the endpoint of AI Vibe Coding is that products write their own code.
Code Structure as Boundary
The product document answers "what to do", the existing code structure answers "where to do it and what to change it into".
These are two sides of the same problem: the document gives the behavior contract, the architecture gives the context boundary. The memory file we give to LLM usually contains the project architecture to help it quickly locate code. When an agent changes the wrong place, most of the time it's not an ability problem but not knowing which code belongs to this feature.
Why can architecture serve as context?
Because code structure is not a physical directory; it's a set of Bounded Contexts already cut by humans. Module boundaries, dependency direction, shared types — these are promises that "changes here won't affect there". For an LLM, these boundaries are pre-cut context slices:
- No need to read the whole repo, just the relevant subgraph;
- Tasks naturally fall into a module, no new exploration each time;
- Change scope has a physical boundary; crossing it produces compile errors, not production incidents.
So "splitting tasks by code structure" is about convergence: from the whole repo to the module, to the file, to the function. Every jump must be evidence-based, otherwise you're guessing.
flowchart LR
A[Full repo<br/>hundreds of thousands of lines] -->|memory file: directory tree + responsibilities| B[Target module<br/>one directory layer]
B -->|module contract: entry/dependencies| C[Entry file<br/>a few classes]
C -->|symbol lookup: references| D[Change point<br/>specific function]
D --> E[blast radius<br/>callers/callees/shared types]
E --> F[crossing boundary gives compile error<br/>not production incident]
Memory file is a map, not an encyclopedia.
The most common mistake with CLAUDE.md / AGENTS.md is to write it as a project encyclopedia. Its only real job is routing: guide the agent to the correct file and correct constraints with minimal tokens. So it should contain "index entries", not "knowledge prose".
| Should include | Should not include |
|---|---|
| Directory tree + one-line responsibility per module | Implementation details of each class |
| Key entry files (startup, routing, data access) | Full API documentation |
| Dependency directions and forbidden cross-layer calls | Conclusions obtainable by reading code |
| Common commands (build, test, single-module verification) | Historical background and evolution stories |
| Naming conventions and directory placement rules | Design descriptions out of sync with code |
A usable memory file roughly looks like:
## Architecture
src/
├── modules/order/ Order domain: place order, payment callback, state machine
├── modules/coupon/ Coupon domain: template, issue, verification (pure computation, no dependency on order)
├── modules/user/ User domain: account, permission, profile
├── shared/ Cross-domain: types, error codes, analytics (only add, never change semantics)
└── infra/ Infrastructure: DB, cache, MQ clients
## Dependency Direction
modules/* → shared, modules/* → infra
Forbidden: modules/a → modules/b direct import; cross-domain must go through shared/contracts
## Commands
pnpm test modules/order Single-module test
Critical rule: dependency direction must be written as prohibitions. "What's allowed" the agent may not remember; "what's forbidden" it will respect, because crossing it can be immediately verified.
Four convergence steps from architecture to task:
- Locate (where to change): map entities and states in the document to modules. No mapping is an architecture gap — either add a module or reuse one, but decide before starting.
- Anchor (where to enter): find entry files — route registration, event subscription, scheduled tasks, external interfaces. An agent reading inward from the entry is an order of magnitude faster than scanning outward from the directory.
- Calculate radius (what else will move): changing a function pulls in callers, callees, and shared types. Files within the radius must be listed in the task scope; outside the radius, explicitly forbid.
- Mark frozen zones (what not to touch): write
shared/, common component libraries, and other domains as frozen. Without frozen zones, agents are likely to "helpfully optimize" a bunch of unrelated diffs.
The resulting task card should carry localization, not just the requirement:
Task: Coupon stacking calculation for order page
Target module: modules/order (new file rules/coupon-stacking.ts)
Reference contract: modules/coupon (reuse calcDiscount, forbidden to write calculation logic directly)
Allowed changes: modules/order/**, shared/contracts/coupon.ts
Forbidden changes: modules/coupon/**, shared/errors.ts (reuse existing error codes)
Acceptance: pnpm test modules/order -- coupon-stacking
This card carries the information of a small architecture spec, but it's more precise than any description because paths and prohibitions can be tool-verified.
When people write code, they are good at being lazy: no comments, no design patterns, so readability worsens. When AI writes code, those lazy places must be written into constraints in advance. It will get faster and better; in the end, programmers will probably really become operators on an assembly line. But the upstream of that assembly line has to be made clear by ourselves — either by speaking clearly or by relaying clearly.