From Prompt to Harness: The Engineering Stack That Tamed LLMs

Foreword

I'm not naturally keen on trying new tools, but my deskmate loves novelty. Last year, they were already enthusiastically recommending AI to me. At the time, I scoffed at it; now, I'm studying it frame by frame 😶‍🌫️.

I remember when AI first emerged, the interaction was simple question-and-answer. I'd consult the AI about a problem, then copy and paste. Last year, my concept of AI programming was still stuck at a souped-up autocomplete, subconsciously assuming it couldn't do real work. Before the New Year, I felt the AI trend hadn't really taken off yet. Although various models were constantly iterating, I was still stuck in that Q&A mode!

After the New Year, somehow, everyone suddenly started using AI like crazy. It was really, so sudden 😲, and the company started pushing it too. Once I started, I couldn't stop. This stuff is genuinely amazing. It's gotten to the point where I can barely write code anymore. Most of the time, I'm just gaslighting the AI into writing features and fixing bugs while I slack off, wait, review, and verify. It has genuinely changed the development workflow.

So, this article mainly records the evolution of AI as I see it. I'll also attach a few classic common questions at the end 🔥.

How Prompts Evolved

Large Language Models (LLMs) have only been around for about three years. Looking back, only two things have changed:

• The models themselves have gotten stronger.
• The constraints we place on them have increased.

The essence of a large model is predicting the probability of the next word. Simplified, it's a massive function y=f(x, θ): input x, output an increasingly accurate y. We won't discuss how the models themselves iterate here (I don't understand it anyway, this stuff goes in one ear and out the other 😂), because most of the time, we are adding constraints to the model. When talking about LLMs, a bunch of terms often pop up:

Prompt -> Tool -> MCP -> Agent -> Skill -> Harness

Each sounds more advanced than the last, and they're often easily confused. In reality, you can understand it as moving from left to right, the contextual constraints on the model increase. The capabilities of large models are indeed getting stronger, but they can't guarantee good execution every time. So, to turn uncertain things into certain things, various constraints and specifications emerged. The feeling it gives me is roughly this: originally, a horse could gallop freely across the vast prairie; now, there are fences, and the horse can only run within a suitable range:

Next, let's use a simple question to deduce the evolution of prompts:

Is today suitable for cycling to work?

1. Pure Prompt

Suppose you ask the model this question directly. An early model might answer:

If the weather is clear and the distance isn't far, today is suitable for cycling to work.

At first glance, it seems fine. But the problem is: does it really know today's weather? Does it know where you are? Does it know when you leave for work? Does it know your commute route? It doesn't know any of this. So, this answer is essentially not a judgment, but a "seemingly reasonable" patch of text based on common sense. The pure Prompt process is roughly as follows:

User: Is today suitable for cycling to work? -> LLM reads context -> Predicts next token based on LLM -> Generates a seemingly reasonable suggestion

What can a pure prompt do?

• Explain what to pay attention to when cycling to work.
• Summarize the pros and cons of cycling.
• Write a commuting suggestion.
• Give some common-sense reminders.

But it cannot truly judge if today is suitable for you to cycle, because this question requires real-time information and personal context.

2. Prompt Engineering

Later, people discovered that although the model is prone to guessing, if we write our requirements clearly, it becomes much more stable. At this point, we would typically change the input to:

You are a commuting advice assistant.
When a user asks a question like "Is today suitable for going out?", do not jump to conclusions directly.
First, determine if information like city, commute time, commute distance, and weather is missing.
If key information is missing, ask follow-up questions first.
If there are no real-time tools, clearly state that you cannot judge real-time conditions.

The model might then answer:

I need to know your city, departure time, and commute distance to judge if today is suitable for cycling. Without real-time weather and route tools, I can only give general advice.

This is much more reliable. The essence of Prompt Engineering is not making the model suddenly know the real world, but preventing the model from improvising too freely. It's like guiding a newcomer at work. If you just say, "Help me see if I can cycle," the newcomer might answer casually. But if you say, "First confirm the city, then check the weather, then look at the commute time, and finally answer with suitable/unsuitable/cycle with caution," the matter becomes much more stable.

A good Prompt generally possesses the following descriptions:

Element	How to write it in this example	Function
Role	You are a commuting advice assistant	Limits the identity of the response
Task	Judge if it's suitable to cycle to work	Clarifies the goal
Missing Info	City, time, distance	Prevents blind judgment
Constraint	State if no real-time data is available	Reduces fabrication
Output Format	Conclusion + Reason + Suggestion	Stabilizes the response

So, this stage also saw the emergence of prompt engineers, prompt collectors, prompt sellers, etc. However, these are all one-off text snippets, difficult to maintain and evolve continuously. Another fundamental problem is: no matter how well-written a Prompt is, the model won't magically know if it's raining today. So, next, Tools appeared.

3. Tool

With Tools, things started to look decent because the model could finally "look it up".

The user still asks: Is today suitable for cycling to work? The model can first judge:

This question requires real-time information; I need to check the weather.

So, it starts calling the weather tool, getting temperature, rain probability, wind strength, air quality, and the weather forecast for the next few hours. Then it answers:

There's light rain this morning with force 4 winds; the roads might be slippery. Cycling is not highly recommended. If you must cycle, it's suggested to wear a waterproof jacket and avoid peak traffic routes.

This is no longer pure guesswork. This is the result of tool query + language organization. The process in the Tool stage is as follows:

User: Is today suitable for cycling to work? -> LLM judges that real-time info is needed -> Calls weather tool -> Returns weather data -> LLM generates commuting advice

This step is crucial. Because from here on, the LLM is no longer just a "talking model" but starts becoming a "model that can connect to the external world." However, Tools also have problems. If you only check the weather, one tool is more than enough; but "cycling to work" clearly needs more than just weather info. It might also need to check:

• Map routes
• Commute distance
• Traffic conditions
• User preferences
• Company clock-in time
• Whether there are shared bike stations

Once there are many tools, problems arise. Each tool has its own interface, parameters, authentication, and return format. If the model has to connect to each one individually, it's going to blow up. Fortunately, the programming field has long had mature solutions for this kind of problem: add an intermediate layer to unify various interface specifications. Thus, MCP made its entrance.

4. MCP

The problem MCP solves is not "Can the model think?". It solves: How can the model stably, uniformly, and securely connect to and call a bunch of external tools? Without MCP, the relationship between the LLM and tools looks like this:

It's like having a pile of devices on your desk: one with a round port, one square, one old-style USB, one proprietary interface, and one that needs a separate driver installed. Every time you connect a new device, it's a hassle. With MCP, it's different; it becomes a universal socket:

Returning to the original question, "Is today suitable for cycling to work?", Tool solves: Can we check the weather, route, calendar? MCP solves: Can these tools be connected in a unified way, discovered, called, and governed? Its benefits are:

Without MCP	With MCP	Change
Each tool adapted individually	Tools exposed in a unified way	Lower integration cost
Model doesn't know what tools exist	Discoverable tools available	Clearer scheduling
Parameter formats vary	More structured input/output	Easier orchestration
Permissions scattered everywhere	Centralized governance possible	More suitable for enterprise deployment

Of course, sometimes we also call internal tools "Tool" and external tools "MCP".

5. Agent

With Tool and MCP, the model can connect to various tools. But the question "Is today suitable for cycling to work?" still can't be solved by calling just one tool. How to better coordinate and schedule these tools? That's where Agent comes in. A decent Agent should break down the task itself:

1. First, confirm which city the user is in.
2. Query the weather for today's commute time.
3. Query the cycling route and distance.
4. Judge if it's raining, windy, or if the air quality is poor.
5. Check the user's calendar for early meetings.
6. Synthesize a suggestion.
7. If cycling isn't suitable, provide alternatives.

At this stage, the model is no longer just answering questions but completing tasks. The Agent's process looks roughly like this:

The core of an Agent: not better at chatting, but better at arranging steps and getting concrete things done. Before, you had to tell the model step-by-step: first check the weather, then the route, then see if I have a meeting, and finally give a suggestion. Now, you just say, "Is today suitable for cycling to work?" and it will decompose the task itself. Of course, Agents also bring new problems: Will it call tools recklessly? Will it access data it shouldn't? What if it fails? What if calling too many tools wastes costs? Who takes responsibility for bad advice? Maybe it makes a mistake midway, you correct it, and it's fine, but next time it might make the same mistake again.

So, an Agent alone isn't enough. We also need to solidify common tasks, encapsulating this context into stable capabilities. This is when Skill appeared.

6. Skill

A Tool is a single tool, MCP is a unified interface, an Agent is an executor that can break down steps on its own. A Skill is more like a packaged, specialized capability, a mature SOP. Judging "Is today suitable for cycling to work?" doesn't just involve calling tools; it also includes a set of stable processes: This is very much like writing code. You can, of course, write it manually every time: first check the weather, then the route, then the meetings, then judge if cycling is suitable. But after writing it many times, you realize: isn't this just a fixed capability? Why not encapsulate it?

The value of a Skill lies right here. It encapsulates recurring tasks into a reusable capability pack. And this capability pack isn't code; it's just an ordinary .md file, with an extremely low barrier to entry. A Skill usually defines when to trigger, what information is needed, how to ask follow-up questions when info is missing, which tools to call, what the judgment rules are, what the output format is, and what behaviors are prohibited. So, a Skill is more stable than a pure Agent. An Agent is like an "improvisational executor," while a Skill is like a "trained standard procedure." Not only that, it also turns one-off prompts into sustainably maintainable documents.

"Writing a Skill is easy, but writing a good Skill is hard." A Skill that can run stably often implies a massive amount of trial and error, edge-case handling, and continuous maintenance. It's still better for professionals to solidify these, as they best understand the boundaries of tool capabilities and the optimal usage in real scenarios, and then share them.

7. Harness

Yes, Harness just appeared like that 😂. It is the foundation that truly allows LLM applications to land. By this point, we've essentially moved from discussing "model capability" to "engineering systems." When it comes to actual deployment, the most troublesome aspects are often not the model itself, but the following questions:

• Can the user's location be used?
• Who manages calendar permissions?
• What if a tool call fails?
• What if results from multiple tools conflict?
• How is user privacy protected?
• Can the model call a ride-hailing tool?
• Can it automatically send messages?
• How are logging and auditing done?
• How are Skills loaded?
• How are MCP Servers managed?

When you add all these together, it approaches a Harness. So, Harness isn't something that just appeared; it has always been there, it just now has a fitting name. From this perspective, "cycling to work" under the Harness lens becomes:

What the end-user sees might just be one sentence:

Today isn't very suitable for cycling. Light rain in the morning, strong winds, and you have a meeting at 9 AM. The subway is recommended. If you cycle, it's suggested to leave 20 minutes early and bring a raincoat.

But behind the scenes, the system did a lot: understood intent, judged what information was needed, checked permissions, called tools, synthesized analysis, generated a suggestion, controlled boundaries, and returned a result. So, building LLM applications now isn't just a competition of "Can the model answer?" More importantly, it's about having a system that allows the model to complete tasks stably, safely, and reusably.

Summary of Evolution

Here's a simple summary of the prompt's evolution, again using "Is today suitable for cycling to work?":

1. Pure Prompt: "If the weather is good, cycling to work is quite nice."
2. Prompt Engineering: "I need to know the city, time, and commute distance first; I can't judge directly."
3. Tool: "I checked, and there's light rain this morning."
4. MCP: "Tools like weather, map, and calendar can be connected in a unified way."
5. Agent: "I checked the weather, route, and calendar, and found you have a 9 AM meeting. The risk of cycling is a bit high."
6. Skill: "I can stably complete the 'commute mode suggestion' task."
7. Harness: "I can complete this task within a system where permissions, security, context, and tool orchestration are all controllable."

The model's evolution isn't from "can't chat" to "can chat," but from a model that predicts the next token, gradually transforming into an intelligent execution unit that can connect to the real world, call external tools, decompose tasks, reuse skills, and be safely hosted by an engineering system. This is somewhat similar to front-end engineering. At first, you write a few pages directly; later, with more pages, you need components; with more components, you need state management; as the project grows, you need engineering; for team collaboration, you need permissions, standards, releases, and monitoring. LLMs are the same. At first, a single Prompt was fun; later, you needed to connect Tools; with many tools, you needed MCP; for complex tasks, you needed Agents; for stable processes, you needed Skills; for real-world deployment, you still need a Harness. After all, what users really want isn't just a pretty sentence.

Here's a table to reinforce memory (the evolution order isn't perfectly accurate, but don't mind the details 😬):

Concept	Analogy	Function	Cycling to Work Example
LLM	Brain	Understands and generates
Tool	Hand	Performs specific actions	Check weather, check route
MCP	Power Strip	Standardizes tool connections	Unifies access for weather, map, calendar
Agent	Executor	Decomposes and advances tasks	Decides what to check first and next
Skill	Specialized Ability	Stable, reusable capability	Cycling commute advice assistant
Harness	Body & Safety System	Hosting, orchestration, governance

Common Questions

Doesn't a Skill still feel like a Prompt?

Looking closely, a Skill doesn't seem that different from the initial prompt. If you write your prompt completely enough, it could also be a Skill. After all, both are .md documents described in natural language, and both are reusable. You can directly stuff all prompt snippets into the context, or you can manually select the needed prompts before feeding them to the large model. Both operations are fine. So, what are the advantages of a Skill?

• Lower token cost
• Less context usage, no redundancy
• Higher cache hit rate
• Sustainable iteration
• More suitable for orchestration

A prompt can carry experience, but it's not suitable for carrying a lot of experience.

Why does "predicting the next token" lead to emergent reasoning abilities?

The essence of a large model is predicting the probability of the next word. The act of "predicting the next token correctly" itself forces the model to learn many structures deeper than surface-level text, enabling it to find intricate relationships behind things from more dimensions. More accurately, in real language data, to consistently predict correctly, the model is forced to learn state tracking, rule composition, causal chains, and intermediate calculation processes hidden behind the text. Reasoning ability can be seen as a byproduct of this high-quality sequence modeling, which is also an effect of quantitative change leading to qualitative change.

If it's probabilistic inference, why does the same input produce different outputs?

Because the model's output is usually not "the single correct next token", but "a probability distribution over a set of candidate tokens". Any information we provide will influence the model's generation result. So, a more accurate description is "in a given context, selecting the most suitable next word from a set of candidates". This selection process leads to randomness in the results. Of course, this is adjustable with a parameter we call Temperature. Temperature is a parameter that controls the "randomness/creativity" of the large model's output:

• Lower temperature: Output is more deterministic, conservative, and tends to be consistent (like a rigid eight-legged essay).
• Higher temperature: Output is more divergent and creative (like a prose poem).

Sometimes I feel that whether AI is used well is also a matter of probability. Using it well means your description fits the AI's appetite. Current optimization is also prompt optimization, which might work well now but could worsen later. A simple model upgrade doesn't guarantee better results than before, and then you need to patch up the prompts, which sounds nicer when called optimization.

Why do models hallucinate?

Hallucination isn't the model "lying"; it's that under its "guess the next word" mechanism, it's forced to generate content it doesn't actually know. A large model is fundamentally a language probability predictor, not a fact database. It always has to output something, even if it has no reliable basis. This is the root of hallucination. So, when it encounters an unfamiliar question, it doesn't stop; instead, it continues to generate the text that looks most like a "correct answer" — sounds right, but might be wrong.

What content is actually sent to the large model with each request?

The large model itself is stateless: it doesn't remember the previous request. So, every time, you have to resend "all the information it needs to see" at once. Suppose you ask an AI: "Summarize the last email for me." The actual content sent to the model might look like this (pseudo-structure):

{
  "model": "xxx-pro", // Model identifier
  "messages": [ // A list of messages, including system prompt && conversation history && current user input
    {
      "role": "system",
      "content": "You are a corporate assistant, follow these rules... Output in Chinese..."
    }, {
      "role": "user",
      "content": "Help me check this week's schedule"
    }, {
      "role": "assistant",
      "content": "You have 3 meetings this week..."
    }, {
      "role": "user", // Your current input is here
      "content": "Summarize the last email for me\n\n[Email body pasted here]"
    }
  ],
  "tools": [ // Optional tools; Tool or Skill definitions go here
    {"name": "get_email", "description": "...", "parameters": {...}},
    {"name": "search_docs", "description": "...", "parameters": {...}}
  ],
  "temperature": 0.3, // Sampling parameter: temperature
  "max_tokens": 1024, // Sampling parameter: max tokens to generate
  "stream": true // Other metadata: whether to stream the response
}

Is there any point in directly training a code-specific model?

Current large models are usually post-trained on a specific domain dataset, built upon an already pre-trained base model, to create a "new" model. So, can we skip the general model and train a small, specialized model using only domain-specific data?

Of course, it's possible. Early AI code completion did just this. But if you only feed it code corpora, the model might be more deterministic in coding, but another problem is that it might not understand normal prompts. So, to improve the model's input understanding, we'd need to continuously expand the base corpora, eventually "training it back" into a general model. Therefore, compared to building a pure coding model, making a general model better at understanding and writing code is usually more cost-effective.

Conclusion

So, how to use AI well:

• Use a good model.
• Provide sufficient context.

The model's own effectiveness is what it is; we usually can't intervene. What we mainly refine is context engineering, using an engineering mindset to solve the LLM's uncertainty, effectiveness, and cost issues. However, there's still a huge gap between individuals. How to better ask questions and describe problems is where experience shows. The ideal state is: Write good prompts, and even a mediocre model can run.

The winds have also shifted now. You don't need to read various source codes anymore (the pace is too fast, you don't have time to read, and you can't get into it). Interviews don't ask about those things anymore. They just ask if you've used AI, how you use it, and how it improves efficiency. If you say you haven't really used it, then the door is to your right. But my biggest takeaway is that although AI makes development easier, it simultaneously compresses the time for requirements, which in turn compresses people's time. It's like watching a video at double speed; you just can't stop 😂 (the time saved is for slacking off, not for rushing to the next requirement).