LLMs Don't Need Better Reasoning — They Need a Heuristic Inventory
When an LLM makes decisions on its own, it can't offer good suggestions and easily goes off track. The core issue isn't insufficient reasoning ability, but a lack of heuristics — it has no "principles" to reason from. Humans steer themselves with principles, theories, and maxims; second-order abstraction is the mechanism for creating principles for LLMs.
How to read this article: The first act is "why LLMs go off track" (diagnosis), the second is "how humans solve this" (deconstruction), and the third is "our mechanism" (the prescription). The three acts form a causal chain — see the full picture first, then dive into the details.
Who this article is for: If you are designing Agent systems, building RAG pipelines, or puzzled by "LLM harness constraints not working well," this article will give you a thinking framework. It's not a "how-to" tutorial; it's about the mechanism of "how to think" — high concept density, long argument chains, but it hits the mark. If you're looking for plug-and-play code or a 5-minute speed read, this article might not be for you (suggest bookmarking it and returning when you encounter real problems). 📌 This article is part of the DialogMesh project series — Second-Order Abstraction is DialogMesh's meta-methodology, responsible for "growing" transferable heuristic rules from behaviors and conflicts. The project is still iterating.
1. A Moment of "Can Choose, But Can't Judge"
You ask an Agent to refactor a module. It gets three candidate solutions:
Solution A — An architecture from a paper (cutting-edge, but heavy)
Solution B — The existing project's pattern (stable, but old)
Solution C — A blend: the idea of A + the skeleton of B
How does the Agent choose? By habit.
Most of the time it will pick C — blending is the safest, it seems to "cover all bases." Sometimes it gets attracted to A — the halo of a cutting-edge paper. If you ask it why, it can give beautiful reasons. But you know: that's not judgment, that's a habitual path.
What should real judgment be?
It's questioning the solutions themselves: "Maybe all three are wrong — perhaps I should first see how other projects handle this, or first confirm whether the refactoring is truly necessary."
Why doesn't the Agent question? Because it can't take the long road.
2. Small Cost, Small Loop: The "Loss Function" of the Agent Cycle
The Agent's execution loop has a hard constraint: prevent infinite loops.
To avoid infinite loops, the system adopts a "small cost, small loop" approach — each decision takes only a small step, verifies, then takes another step. This is very much like a loss function: every step has a cost, and costs cannot accumulate indefinitely.
The cost of taking the long road:
Question the solution → read more papers → discover more solutions → question again → ...
→ unbounded divergence → infinite loop / budget overrun
The benefit of the small loop:
Each step is bounded → verifiable → reversible → system stability
So the Agent is trapped in a dilemma:
Doesn't dare to go far (loop constraint) × Can't judge (habitual decision-making)
Relax the loop constraint? No — that leads to infinite loops. Give the Agent stronger reasoning? Also no — no matter how strong the reasoning, without callable judgment criteria, it still picks from three solutions by habit.
The cure is not relaxing the loop, but stuffing "principles" into the loop.
3. How Do Humans Solve This Problem?
Faced with the same situation, humans invoke heuristics:
"Refactor early, save trouble later" (a variant of YAGNI)
"Three repetitions deserve an abstraction" (Rule of Three)
"Let the tests speak first" (Test-First)
"Cutting-edge solutions should wait for project validation" (Technical debt awareness)
"Question the problem first, then the solution" (Metacognitive questioning)
These are not logical deductions, not exhaustive searches, but compressed judgment criteria — transferable, independent of specific scenarios, enabling fast decisions.
3.1 Ecological Rationality: Heuristics Are Not Biases, but Environment-Adapted Rationality
Gerd Gigerenzer's research on Ecological Rationality is the theoretical foundation for this view. Traditional decision theory treats heuristics as "cognitive biases" (Kahneman's System 1 bias perspective), but Gigerenzer refutes this framework:
Logical Rationality: Heuristic = Bias (suboptimal compared to the optimal solution)
Ecological Rationality: Heuristic = Environment-adapted rationality (efficient and accurate in uncertain environments)
Key point: Heuristics are not "a compromise when you can't compute," but "using environmental structures for fast decisions"
→ A few heuristics + environmental structure = good decisions
→ Exhaustive search actually performs worse in uncertain environments (overfitting to current information)
This is the theoretical origin of the "heuristic inventory" concept — research focuses on the repertoire of heuristics possessed by individuals/institutions, directly corresponding to the article's "expert = a large set of callable heuristic patterns."
3.2 Expert Intuition: Not Deep Calculation, but Pattern Recognition
Adriaan de Groot's (1946) classic research on chess masters supports the "pattern inventory" view:
de Groot's finding: Masters vs. average players
→ The difference is not in "calculating more moves" (both have similar look-ahead depth)
→ The difference is in pattern recognition: Masters recognize the board at a glance and invoke corresponding response patterns
2011 review study: 70 years of subsequent research supports de Groot's findings
→ Consistent with predictions of pattern recognition models
Real-world corroboration: Medical diagnosis — experienced doctors make quick judgments based on symptom patterns,
not by checking against all possible diseases one by one (the same "pattern inventory" mechanism)
Expert = a large set of callable heuristic patterns. Humans steer themselves with these compressed artifacts — "principles, theories, maxims" — facing a new situation, invoking a suitable principle, rather than reasoning from scratch.
What LLMs lack is not reasoning ability. What they lack is a heuristic inventory.
🧠 Section Recap: What do humans rely on for judgment? Not exhaustive search, but compressed, transferable criteria.
4. Constraints = A Dynamical System, But Constraints Can Also Go Wrong
When we design systems, we habitually use constraints to guide behavior:
Permission constraints: Cannot write to /etc
Security constraints: Shell commands require approval
Loop constraints: Maximum steps / budget cap
Constraints are like a potential field in a dynamical system — they don't prescribe a trajectory, but they guide behavior within a legal region. Without constraints, LLM behavior is a free particle; with constraints, behavior is anchored in a reachable region.
But constraints themselves can also go wrong:
Wrong constraint → Blocks correct behavior (over-restriction)
Outdated constraint → The environment has changed, but the constraint remains in the past
Conflicting constraints → Two constraints contradict each other, behavior is at a loss
Too few constraints → goes off track. Too rigid constraints → becomes paralyzed. More constraints are not always better; constraints themselves need to be "re-constrained."
5. Metacognition: Thinking About Thinking, Re-constraining Constraints
When constraints themselves go wrong, what is missing?
Metacognition — the organizing and reviewing of thinking itself.
But there's a trap here: "Letting the LLM double-check its own answer" ≠ Metacognition.
That's ReAct — running the same thinking again, no new information, just self-confirmation. True metacognition is:
Re-constraining behavior under constraints:
Which heuristic was used for this decision? Why this one and not another?
Is this heuristic still applicable? When was it last used?
Is there a better heuristic I haven't thought of?
This is "organizing and reviewing thinking itself" — not re-reasoning, but auditing what principles you used.
5.1 Psychological Evidence: Metacognition is the Cornerstone of Learning and Decision-Making
"Metacognition ≠ self-checking" is not our slogan; it's an established conclusion from psychology and neuroscience:
Metacognition Definition:
An individual's ability to evaluate and monitor their own cognitive processes (perception/memory/decision-making)
Post-decisional reflection:
Decision-making is not "done once a choice is made" — humans continuously evaluate and update decisions
→ This is precisely the psychological term for "auditing what principles you used"
Research consensus: Metacognition is the cornerstone of good learning, decision-making, and adaptive reasoning
ReAct's self-checking stops at "re-examining the output" (same-layer loop); metacognition is "auditing the basis of the decision" (cross-layer reflection) — this is exactly the recursive structure to be unfolded in §8.
But a question arises: Where do so many constraints come from?
6. Second-Order Abstraction: Constraints Are Grown, Not Written
This is our method — Second-Order Abstraction.
Not hand-writing constraints, but growing constraints from behavior.
Forward compression: 1000 decisions → clustering → condensed into 10 heuristics
Reverse verification: 10 heuristics → back-propagate to 1000 decisions → what's the coverage?
60-80% = Learned (transferable patterns)
100% = Overfitting (memorized the original text)
0% = Hallucination (the heuristic is fabricated)
Failure → Multi-perspective adjustment (re-examine from four perspectives: structural/semantic/temporal/counter-example)
Pass → Enter heuristic inventory
Why 60-80% and not 100%? 100% means the rules perfectly replicate the training data — that's memorization, not abstraction. 60-80% means the rules capture the main patterns while preserving generalization space. This aligns with the design philosophy of loss functions: stop training before the overfitting point.
This is "pseudo-second-order abstraction" — not "abstracting once more" (which would hollow out the abstraction), but "reverse-verifying the quality of the abstraction":
DMN Divergence: Let LLM guess without constraints → massive hypotheses (including wrong ones)
ECN Convergence: Filter with context → keep aligned hypotheses + reasons for rejection
Heuristic Chain: Condense into transferable heuristic units
Key point: Rejected hypotheses and their rejection reasons are themselves knowledge —
"What was rejected, and why" = learning the boundaries of knowledge
6.1 How to Implement: Divergence is "Masking," Convergence is "Exposing"
The idea is there, but what's the implementation? The engineering implementation of second-order abstraction = the masking and exposing of context.
Divergence: Let the LLM Guess in an "Amnesiac" State
Implementation: Mask context → LLM(temperature=0.8, no_context) → Generate K guesses (K=3-5)
Why is masking useful?
Masking = Depriving evidence → LLM is forced to retrieve pre-training knowledge (priors)
→ Produces divergent hypotheses (including "reasonable but unverified" ones, even wrong ones)
→ These hypotheses are not answers, but candidate "principles"
Example (Kalman × Information Theory — a collision of two opposing philosophies):
"A low-probability event repeatedly appears in the system."
After masking context, let the LLM guess: What is this event?
Guess 1: Sensor noise → should be smoothed out (Kalman-style: low-weight processing)
Guess 2: A new state mutation → should be dug into (Information Theory-style: rare = high information)
Guess 3: A new user intent → should be recorded (Behavioral-style: worth modeling)
→ Three guesses = three candidate "principles," each with a philosophical origin
Why can Kalman and Information Theory, "completely opposite," be unified? This is exactly what divergence aims to expose:
Kalman: Low probability = Noise → Smooth out (maintain state stability)
Information Theory: Low probability = High information content (I = -log₂ P) → Preserve (rare ≠ irrelevant)
Surface contradiction, deep unity (A7):
The judgment basis is not "probability level," but "can it be explained by the existing model":
Explainable → Noise (smooth out, Kalman's R estimation)
Unexplainable → New information (dig deep, Shannon's information value)
→ The same low-probability event, two philosophies give opposite actions,
but are unified under the single criterion of "explainability"
Letting the LLM guess this unification process is far more effective than directly telling it "you should judge by explainability" — because guessing forces it to retrieve the full context of both philosophies, then let them collide in the divergence space.
Convergence: Let the LLM Filter in a "Full Memory" State
Implementation: Expose context → LLM(temperature=0.1, full_context) → Filter + Rank
Why is exposing useful?
Exposing = Restoring evidence → LLM verifies guesses with real data
→ Keeps aligned hypotheses + provides rejection reasons
→ Rejection reasons = Knowledge boundaries ("Guess 1 rejected, because the last three occurrences were all real bugs")
Counterfactuals: Reverse-Pushing Low-Probability, High-Value Events
Divergence isn't just forward guessing; it can also push counterfactually:
Counterfactual:
"If this low-probability event is not noise, what mechanism could cause it?"
→ Reverse the "high value" of Information Theory → push, guess, search, connect
→ Not waiting for it to appear (passive), but actively constructing "what would it mean if it appeared" (active)
Implementation:
Information Theory positioning: Low probability, high value → Mark as to-be-explained (preserve, don't smooth)
Counterfactual divergence: Construct hypothesis "If X is true → what chain reaction?" → K conjectures
Evidence connection: Search for traces of these chain reactions in real data → Hit → hypothesis upgraded
Miss → hypothesis downgraded (but rejection reason enters knowledge boundary)
This is like a detective first hypothesizing "If the murderer is X, what should be at the scene," then going back to look. Counterfactuals allow the system to actively explore even with insufficient information, rather than waiting for data to speak.
Complete Implementation Flow (Real Parameters from Design Doc v5)
Expand to see 7-step implementation parameters (N/K/temperature, etc.)📐 Implementation Parameter Pack (Flexible Reading) — Non-deep-implementation readers can skip this box without affecting the main argument.
⚙️ 2026-08-07 Implementation Calibration: The main trigger has been changed from "timed (every N rounds)" to change-driven — Decision failure / User correction / Axiom conflict / Heuristic obsolescence / Sense of missing axiom → Trigger re-abstraction; Timed distillation is only a fallback (trace≥10). Parameters (N/K/temperature/masking-exposing) remain unchanged.
Every N rounds (N=5):
1. Extract the derivation chains of the last N rounds (L2 entity edges + L2.5 belief trajectories)
2. Divergence: LLM(temperature=0.8, no_context) → K guesses (K=3-5)
└─ Mask context → Retrieve priors → Divergent hypotheses (including Kalman/Information Theory collisions)
3. Convergence: LLM(temperature=0.1, full_context) → Filter + Rank
└─ Expose context → Evidence verification → Keep + Rejection reasons
4. Counterfactuals: For low-probability, high-value hypotheses → Reverse-construct chain reaction conjectures → Find evidence connections
5. Heuristics: LLM generates heuristic chains (pattern description + applicability conditions + counter-examples + reasoning path)
6. Store in heuristic pool, replace old heuristics with low coverage
7. Metacognitive monitoring of coverage: < threshold → Re-diverge; new-old conflict → Adjudicate
Why is "masking/exposing" the core of the implementation? Because it's the only mechanism that allows the LLM to simultaneously invoke "priors (pre-training knowledge without context)" and "posteriors (evidence verification with context)" — priors generate candidate principles, posteriors test principles, counterfactuals expand principles. All three steps are LLM calls, but the masking/exposing switch determines whether it invokes priors or posteriors. This is the engineering realization of "pseudo-second-order abstraction": abstraction is not extracted from data, but generated by letting the LLM collide repeatedly under two information states.
🧠 Section Recap: How is second-order abstraction implemented? A masking/exposing pipeline.
7. The Closed Loop: Behavior → Heuristics → Metacognition → Steering
Now let's string the entire chain together:
Behavior as first-class citizen (A9): Behavior is learning data — your every decision is raw material
White-box (A19): Decisions are viewable and modifiable — you see what principles the system is using
Second-Order Abstraction (A24): Refine heuristics from behavior — constraints are grown, not written
Metacognition (A10): Audit which principle is used — re-constraining constraints
Steering: The heuristic inventory enables judgment-like choices within the small loop
Without second-order abstraction, metacognition is idling — ReAct-style self-indulgence, with no constraint inventory to audit. With second-order abstraction, metacognition has raw material — it audits not the vague "quality of thinking," but the concrete "which heuristic was used this time, why, and is it still applicable."
8. Conflict is Raw Material: Recursive Metacognition
Now we reach the most critical step — abstracting second-order abstraction itself again.
8.1 Heuristics Will Conflict; Conflict is Not a Defect, but Input for the Next Layer
Conflicts are inevitable in the heuristic inventory:
"Blend-first" (safe) vs. "Question-first" (dig deep)
"Smooth noise" (Kalman) vs. "Dig into rarities" (Information Theory)
"Fast feedback" (A16) vs. "Verify truth" (A18)
How to handle conflicts? Can't discard them — conflict itself is information.
Axioms can give birth to theorems, but axioms don't have to be uniform. In our system, A1-A25 are parallel axioms; when they conflict, they are orchestrated by meta-rules (§5):
Uninterrupted experience > Single accuracy (A16 priority)
True verification > Good-looking metrics (A18 priority)
Security constraints are non-negotiable (A21 baseline)
Records are never deletable (A17 baseline)
Fallback: Return to the constraint space (A12 fallback)
This set of meta-rules is itself a "second-order abstraction" of axiom conflicts — orchestration rules condensed from the experience of axiom collisions.
8.2 The Act of Orchestration Choice is Itself a Thinking Behavior
Choosing "experience-first" or "verification-first" — this choice itself is also a behavior.
Layer 1: Behavior → Second-Order Abstraction → Heuristics (constraints grow)
Layer 2: Heuristic conflicts → Meta-rule orchestration → Adjudication (axioms give birth to theorems)
Layer 3: Meta-rule conflicts → Re-abstraction → Meta-meta-rules (adjudicating the adjudication)
Layer 4: ...
This is the true meaning of "thinking about thinking" (distinct from ReAct):
ReAct: Thinking → Check thinking (same layer, no new raw material)
Recursive: Thinking → Abstract the laws of thinking → Use laws to steer thinking
→ Abstract "the use of laws" → ... (each layer produces genuinely new things)
8.3 Recursion Has a Bottom; It's Not Infinite
You might ask: Won't this lead to infinite recursion? No, and this "bottom" is precisely the reverse verification of A24:
Condensation = Compression = Output must be < Input
A dozen axioms → can only condense into 2-3 meta-rules (if there are truly condensable patterns)
→ or 0 (if the axioms are inherently independent, with no conflict patterns to refine)
Not being able to condense more = Normal! The purpose of condensation is not "to produce more rules,"
but "to discover relationships between rules" (orchestration, not stacking).
The bottom of recursion is an adaptive stopping condition:
Bottom = Input diversity is insufficient, condensation benefit → 0, stop
But note: The raw material for condensation is not the axioms themselves, but "instances of axiom conflicts":
Axioms = Constitution (few)
Conflict instances = Case law (infinite! Every real conflict is a new scenario)
Meta-rules = Judicial interpretation (condensed from case law)
→ There are only a dozen axioms, but instances of axiom conflicts are infinite
→ Condensation won't dry up: the raw material is "cases of conflict," not "the list of axioms"
→ The bottom of recursion = when the diversity of "conflict instances" no longer provides new patterns
8.4 One Mechanism, Three Application Facets
Second-order abstraction is not a tool, but a way of thinking — one mechanism, three objects:
① RAG cluster compression → Object: Content → Solves clustering + high-value recall
② Heuristic usage steering → Object: Behavior → Solves LLM decision drift
③ Axiom conflict adjudication → Object: Adjudication → Solves conflicts of meta-rules themselves
Each layer uses the same "masking/divergence/convergence/counterfactual" mechanism, just on different objects: content, behavior, adjudication. The output of each layer is the input for the next — this is recursive metacognition.
Behavior → Heuristics (abstraction)
Heuristics → Meta-rules (abstraction of abstraction)
Meta-rules → Meta-meta-rules (abstraction of abstraction of abstraction)
...until condensation benefit approaches zero (adaptive stop)
8.5 Second-Order Abstraction in RAG: Hot-Cold System × Divide-and-Conquer Retrieval
Among the three application facets, RAG is the most visible one — but it's not an isolated retrieval trick; it's the projection of the hot-cold system (A15) onto the retrieval side.
First, look at the hot-cold layering of memory:
Memory is not stored uniformly; it has "temperature" (A15):
Hot (active) — Current session, fully in memory
Warm (paused) — Recent sessions, persisted, loadable on demand
Cold (cold) — Historical sessions, with decay weights
Frozen (archive) — Compressed, metadata + summary
Temperature is not a single timeline, but a multi-factor composite field:
Time (recent ≠ important) × Access count (LRU) × Topic activity × Behavioral inertia
Temperature manages the timeline; value manages the scarcity axis — two dimensions, not one:
Looking only at temperature loses high-value information that is "rare but cold" (A15)
Looking only at value ignores the retrieval cost of "high-frequency but mundane" items (A7)
Two-dimensional retention strategy:
Temperature dimension (timeline): Hot → Warm → Cold → Frozen (downgrade changes representation)
Value dimension (scarcity axis): I(x) = -log₂ P(x) (rare ≠ irrelevant)
Combined:
High temperature + High value → Full retention, fastest path
High temperature + Low value → Cool down after session ends
Low temperature + High value → Frozen retains milestone summary + index (retrievable)
Low temperature + Low value → Compress to summary, reduce representation dimension
Cooling down is not deletion, but changing representation (this is key):
Frozen is not discarded — high-value content is retained as "milestone summary + index,"
low-value is compressed to summary. Temperature determines the "granularity injected into context,"
not "whether it exists." Echoes A2 granularity scaling.
→ What role does second-order abstraction play here?
It is the "mechanism of cooling down":
Cluster condensation = Changing representation from Cold to Frozen (10000 items → 10 heuristic chains)
Counterfactuals = Semantic awakening from Frozen back to Hot (BGE > 0.8 directly back to Hot)
The blind spot of ordinary RAG = No temperature, and no value axis:
Blind spot 1: High-frequency content is repeatedly retrieved — "How to configure API key" asked 100 times, each time a full search
(No cooling down, everything is Hot, retrieval cost is constant)
Blind spot 2: Rare, high-value content is drowned by similarity — a root cause analysis that appeared only once,
semantically far from any query, never recalled
(No value axis, low-probability entries discarded as outliers)
Second-order abstraction RAG = Hot-Cold × Divide-and-Conquer:
Fast path (Hot): Heuristic chain matching — high-frequency clusters, instant response (no need to search original text)
Deep path (Cold): Counterfactual awakening — rare, high-value entries, recalled via hypothetical connections
How is high-value recall achieved — counterfactuals are the mechanism for "semantic awakening":
Ordinary RAG: Query → Similarity ranking → Take top-k
→ Rare, high-value entries (semantically distant) don't make it into top-k → Never recalled
(BGE similarity < 0.8, cannot awaken)
Second-order abstraction RAG: Query → Two parallel paths
Fast path: Heuristic chain matching (high-frequency clusters, instant response)
Deep path: Counterfactual awakening (low-probability, high-value entries)
Counterfactual: "If this rare entry is related to the current problem, what should it be related to?"
→ Construct K association conjectures → Use conjectures as queries to re-retrieve → Find connections
→ Rare entries are recalled via "conjecture bridges" (not by similarity, but by hypothetical connections)
→ This is the semantic awakening of A15: relevance breaks the time/similarity barrier, directly back to Hot
Why is "rare ≠ irrelevant" the mathematical root? Shannon's self-information I = -log₂ P — the less frequent an event, the more information it carries. One instance of "server crashed, can't find root cause" is more valuable than a hundred instances of "daily coding." Ordinary RAG's similarity ranking discards high-information entries as outliers; the hot-cold system marks them as "low temperature, high value" — Frozen retains the summary, counterfactuals awaken when needed.
Ordinary RAG: Similarity top-k → No temperature, no value axis → High-frequency fully searched + rare drowned
Second-order abstraction RAG: Hot-cold layering (temperature) + Divide-and-conquer (value) → High-frequency instant response + rare can be awakened
This is why "a way of thinking is not a tool": A tool is a one-time compressor; a way of thinking is a recursive mechanism — compress, use the compression, compress "the use of compression." RAG uses it (hot-cold layering + divide-and-conquer recall), condensed heuristics use it (steering decisions), and the choice of axiom conflicts also uses it (adjudication orchestration). The hot-cold system is the storage side of second-order abstraction; second-order abstraction is the retrieval side of the hot-cold system — two sides of the same coin.
🧠 Section Recap: Conflict is not a defect, but raw material for the next layer of abstraction.
9. This is Not "Skill" in Disguise
Some might ask: Isn't this just a skill? Isn't it just storing and reusing successful experiences?
No. First, look at the diagram — the ecological niches of the four mechanisms:
Second-order abstraction is in the top right: High transferability (cross-scenario) × High judgment depth (decides "should it be done")
Skill is in the bottom left: Low transferability (bound to scenario) × Low judgment (only decides "how to do it")
Prompt is in the bottom right: High transferability but low judgment (one-time instruction, no accumulation)
Constitutional AI: Medium transferability × Medium-high judgment (hand-written principles, stable but rigid)
The difference lies in four areas:
| Dimension | Skill | Second-Order Abstraction |
|---|---|---|
| Object | Executable action sequence (how to do) | Transferable judgment criteria (how to think) |
| Verification | Execution success = Effective | Reverse coverage 60-80% = Effective |
| Purpose | Reuse process | Steering decisions + Feeding metacognition |
| Implementation | Template + Parameterization | Masking/Exposing switch (Divergence/Convergence) + Counterfactuals |
Skill answers "how to do this thing," second-order abstraction answers "should this thing be done, and by what principle to judge." The former is a library of procedures; the latter is a value system.
Thought + Implementation: The thought of second-order abstraction is "constraints grow from behavior"; the implementation is "masking→divergence→convergence→counterfactuals→heuristic chain." Just like RPC — the thought is "remote calls like local calls," the implementation can be REST, gRPC, message queues, chosen by environment. Our implementation chose "context masking/exposing": it lets the LLM switch between no-context (prior divergence) and with-context (evidence convergence), actively expanding with counterfactuals — this is one currently feasible way; if a better mechanism comes in the future, the thought remains unchanged.
Humans are also divided this way: skill is "knowing how to fix a car," principle is "checking what's broken before fixing." Skill lets you do the work; principle lets you do the right work.
10. Horizontal Comparison with Existing Ideas
| Idea | Core | Difference from Second-Order Abstraction |
|---|---|---|
| Prompt Engineering | Give model instructions via prompts | Second-order abstraction gives the model transferable judgment criteria, not one-time instructions |
| RAG | Retrieve similar content for context | Second-order abstraction compresses out patterns, not retrieving original text |
| ReAct | Thought→Action→Observation loop | Second-order abstraction is accumulation outside the loop — precipitating heuristics after each loop |
| Constitutional AI | Hand-write a set of principles to constrain the model | Second-order abstraction's principles are grown from behavior, not hand-written |
| Skill / Workflow Template | Reuse successful action sequences | Second-order abstraction reuses judgment criteria, and verifies with reversibility |
10.1 Constitutional AI: The Closest Neighbor, and the Most Critical Difference
Anthropic's Constitutional AI (CAI) is the most famous engineering practice of "constraining LLMs with a set of principles" — guiding and constraining model outputs through an explicit, transparent set of values (a "constitution"). It shares a premise with second-order abstraction: reasoning alone is not enough; explicit judgment criteria are needed.
But the difference is critical:
Constitutional AI: Principles are hand-written (the Anthropic team writes the constitution)
Research direction: How to automatically discover principles / How to evaluate principle effectiveness (in progress)
Second-Order Abstraction: Principles are grown from behavior (behavior → heuristics, reversibly verifiable)
Difference: Hand-written = one-time, dependent on the author's wisdom
Grown = continuous, evolves with behavior, verified by coverage
In one sentence: CAI's constitution is written by "founding fathers"; second-order abstraction's constitution is accumulated from "case law." The former is stable but rigid; the latter is flexible but has a cold-start problem — the two can complement each other: cold-start with hand-written seeds, automatically supplement as experience accumulates.
10.2 LLM-Generated Heuristics: What the Frontier is Doing
"Letting LLMs generate their own heuristics" is not our fantasy — this is a frontier direction in current AI research, with multiple frameworks already doing it:
PathWise: Formalizes heuristic generation as a sequential decision process
→ LLM self-evolutively designs heuristics
HSP (Heuristic Search + LLM): LLM as a reasoning and optimization engine
→ Rationalizes heuristic plans + corrects errors during execution
Verification direction: Using LLMs (ChatGPT, etc.) to verify and classify heuristic thinking stimulus materials in decision experiments
The difference between second-order abstraction and them: PathWise/HSP let LLMs generate heuristics, but the verification mechanism differs — they verify by execution success; we verify by reversibility (reverse coverage 60-80%); and second-order abstraction simultaneously uses heuristics for "steering decisions" and "feeding metacognition," a complete cognitive loop rather than a single-point tool.
11. Known Limitations
Second-order abstraction is not a silver bullet:
1. Heuristics may become outdated — the environment has changed, but the principles in the inventory still use old logic
→ Requires periodic metacognitive auditing (heuristic liveness check)
2. Heuristic inventory cold start — without enough behavioral experience, the inventory is empty
→ Cold start uses built-in seeds (a few hand-written constraints as a transition),
as experience accumulates, hand-written constraints gradually retire
⚙️ **2026-08-07 Implementation Calibration**: Seeds ≠ Axiom list — project axioms (like A1-A25)
are "refined products"; using them as seeds creates a self-confirming loop (axiom→seed→refine from behavior→confirm axiom).
Seeds = **Cognitive structure templates** (phenomenon→origin→boundary→operationalization) + demonstration samples few-shot
(LLM generation specification); Quality criterion: Anchored in formal science constraint space (law of excluded middle/mapping morphology/
probability axioms) → Foundational = Transferable = Low risk of obsolescence.
3. Reversibility has a cost — reverse verification requires retaining behavioral records (A17 records are never deletable)
→ Storage and abstraction are a pair; you can't have abstraction without records
4. Diminishing returns of recursion — the deeper the layers, the fewer new patterns can be condensed
→ Adaptive stop: stop when condensation benefit approaches zero; don't recurse for recursion's sake
Conclusion
When an LLM makes decisions on its own, it can't offer good suggestions and easily goes off track — this is not a problem of reasoning ability, but a lack of heuristics.
Humans solve this problem with principles, theories, maxims — those compressed, transferable, verified judgment criteria. Second-order abstraction is the mechanism for creating principles for LLMs: refining heuristics from behavior, verifying quality with reversibility, using the heuristic inventory to steer decisions within the small loop, and using metacognition to audit "which principle was used this time."
Conflict is not a defect of second-order abstraction — conflict is the raw material for the next layer of abstraction. Axioms can give birth to theorems, but axioms don't have to be uniform; the orchestration of axiom conflicts is itself a thinking behavior, and can be abstracted again. Recursion has a bottom (stop when condensation benefit approaches zero), but every real conflict feeds the next layer.
Constraints are grown, not written. Just as human values are not a complete set from birth, but are precipitated by experience.
Glossary
| Term | Definition |
|---|---|
| Heuristic | Compressed, transferable judgment criteria, independent of specific scenarios |
| Heuristic Inventory | The set of heuristics callable by the system, the machine version of expert intuition |
| Small Cost, Small Loop | Execution constraint to prevent infinite loops: each step bounded, verifiable, reversible |
| Second-Order Abstraction | Refining heuristics from behavior, verified by reversibility (coverage 60-80%) |
| Pseudo-Second-Order Abstraction | Not "abstracting once more," but "reverse-verifying the quality of abstraction" (DMN divergence→ECN convergence→Heuristic chain) |
| Masking/Exposing | Implementation mechanism: no-context divergence (invoke priors) + with-context convergence (verify evidence) |
| Counterfactual | Reverse-pushing low-probability, high-value events: construct "if X is true→chain reaction" conjectures, then find evidence connections |
| Reversibility | The proportion of original content that the compressed product can inversely restore; 60-80% = learned |
| Metacognition | Organizing and reviewing thinking itself, re-constraining constraints |
| Recursive Metacognition | Conflict→Meta-rules→Meta-meta-rules... each layer is the input for the next, stop when condensation benefit approaches zero |
| Constraints as a Dynamical System | Constraints don't prescribe a trajectory, but guide behavior within a legal region (potential field) |
| Steering | Using heuristics + constraints to guide the LLM to make judgment-like choices within the small loop |