LLMs Don't Execute Code: The Runtime Does

Many AI products easily create an illusion:

It seems like it can actually "do work."

It can search the web, analyze Excel files, check the weather, read local files, and even help us modify code, run commands, and operate browsers. At first glance, it feels magical, as if the large model has suddenly stepped out of the chat box and truly touched the external world.

But from a developer's perspective, it's not that mysterious.

The large model itself cannot actually open a browser, access a database, or execute a function on its own.

What truly transforms AI from "being able to answer" to "being able to do things" is Tool Use, also known as tool calling.

Where is the LLM Trapped?

First, let's dismantle a misconception: an LLM itself is not a program that actively operates on the world.

Its core capability is still predicting the next token based on context. It excels at understanding, reasoning, and generating text, but it inherently has several limitations:

It doesn't know real-time information
It cannot directly read local files
It cannot directly query databases
It cannot directly call business APIs
It cannot execute code by itself

For example, if a user asks:

What is the closing price of Tsingtao Beer?

If it relies solely on the model's own parameters, it's difficult to give a reliable answer.

Because stock prices change, real-time data won't be stably stored in the model's parameters. The model can guess an answer based on its training data, but this guess lacks engineering credibility.

Therefore, it needs tools.

The core idea of tool calling is:

The model is responsible for judging "this question requires looking up data," and the program is responsible for actually looking up the data.

This is the foundation upon which many Agent products work.

Tools Are Not Magic, But a Manual

The model doesn't inherently know what functions exist in a project.

Developers must first describe the tools to the model, letting it know:

What the tool is called
What the tool can do
When to use it
What parameters are needed
What types the parameters are
Which parameters are required

In code, this is usually written as a tools configuration.

For example, a tool for getting a stock's closing price:

const tools = [
  {
    type: 'function',
    function: {
      name: 'get_closing_price',
      description: 'Get the closing price of a specified stock',
      parameters: {
        type: 'object',
        properties: {
          name: {
            type: 'string',
            description: 'Stock name'
          }
        },
        required: ['name']
      }
    }
  }
];

The truly interesting part of this code is not that it writes a JSON.

It's that it translates a function from the traditional software world into a language the model can understand.

The model doesn't understand your backend implementation, how the database queries, or how the stock API is wrapped. But it can read the sentence "Get the closing price of a specified stock" and understand that the parameter name is the "stock name."

This step can be understood as a kind of "cognitive implantation": writing the capabilities of external tools into the context of the model's current conversation.

The First Call: The Model Is Only Making a Decision

After having the tool description, when the user asks again:

What is the closing price of Tsingtao Beer?

The program sends the user message and the tool configuration together to the model:

async function sendMessage(messages) {
  const res = await client.chat.completions.create({
    model: 'deepseek-v4-pro',
    messages,
    tools,
    tool_choice: 'auto'
  });

  return res;
}

Note the tool_choice: 'auto' here.

It doesn't mean "the model automatically executes the tool," but rather "the model can decide for itself whether to call a tool."

If the model determines that this question requires looking up the stock closing price, what it returns might not be ordinary text, but a structure like this:

tool_calls: [
  {
    type: 'function',
    function: {
      name: 'get_closing_price',
      arguments: '{"name":"青岛啤酒"}'
    }
  }
]

In this step, the model does three things:

Recognizes user intent: the user wants to check a stock's closing price
Selects a tool: it should use get_closing_price
Generates parameters: the stock name is "Tsingtao Beer"

But it hasn't actually retrieved the price yet.

Because the model cannot execute functions.

It merely generated a structured request saying, "I want to call this tool."

The Runtime Is the One That Actually Does the Work

The one that actually executes the function is the runtime.

That is, Node, Python, Java, or other backend code.

In this example, after the program receives the tool_calls returned by the model, it parses the parameters and then calls the local function:

function get_closing_price(name) {
  if (name === '青岛啤酒') {
    return '67.92';
  } else if (name === '贵州茅台') {
    return '1488.21';
  } else {
    return 'Stock not found';
  }
}

The execution process is roughly:

const toolCall = response.choices[0].message.tool_calls[0];
const args = JSON.parse(toolCall.function.arguments);
const price = get_closing_price(args.name);

This step is the real "operation on the external world."

If this were a real project, inside this function could be:

Querying a database
Calling a third-party API
Reading a local file
Executing a search
Invoking an internal business service

The model is only responsible for saying, "Which tool do I want to use, and what are the parameters."

The runtime is responsible for actually doing it.

The Tool Result Must Be Handed Back to the Model

After the tool executes, the result isn't just crudely thrown at the user.

A more complete approach is to put the tool result back into the conversation as new context:

messages.push({
  role: 'tool',
  content: price,
  tool_call_id: toolCall.id
});

The tool_call_id here is crucial.

It tells the model: this tool result corresponds to which previous tool call.

If multiple tools are called in a single request, like checking the weather, schedule, and inventory simultaneously, without this id, the model would easily confuse which result belongs to which question.

After putting the tool result back into messages, call the model again:

const finalRes = await sendMessage(messages);

Now the model finally possesses the real data.

It no longer needs to guess "what is the closing price of Tsingtao Beer," but can organize a natural language response based on the 67.92 returned by the tool.

So a complete Tool Use is not a single model call, but a chain:

User asks a question
-> Model decides to call a tool
-> Runtime executes the tool
-> Tool result is written back to context
-> Model generates the final answer

What's the Difference Between Tool Use and a Regular API Call?

At this point, a question might arise:

Isn't this just an API call?

It's a bit similar, but not exactly the same.

A regular API call usually has a programmer pre-writing a fixed flow:

User clicks a button -> Calls a specific API -> Displays the result

The special thing about Tool Use is that there's an extra layer of model decision-making in the middle:

User asks in natural language -> Model judges intent -> Model selects tool -> Program executes tool -> Model organizes answer

In other words, tool calling doesn't give the model "the ability to execute functions," but rather lets the model participate in the judgment process before a function call.

It can decide based on natural language:

Whether to call a tool
Which tool to call
How to fill in the parameters
How to interpret the tool result for the user

This is where the real value lies in combining LLMs with traditional programs.

Traditional programs are responsible for stable execution, and large models are responsible for understanding intent and organizing language.

The Clearer the Tool Description, the More Stable the Call

Tool Use is powerful, but it won't work stably just by casually writing a function name.

If the tool description is vague, the model can easily choose the wrong tool.

If the parameter design is chaotic, the model can easily pass the wrong parameters.

For example, if a parameter is called query, but it doesn't specify whether it's a city name, stock name, or search keyword, the model can only guess.

Tool design also requires engineering capability:

Function names should be clear, e.g., get_weather is better than handle
The description should be specific, telling the model when to use it
Parameter types should be accurate, don't just write everything as string
required should be explicit, don't let the model miss parameters
Return results should be clean, making it easy for the model to continue understanding
High-risk operations should require confirmation, don't let the model execute them directly

The more a tool resembles a clear interface contract, the more stable the model's calls will be.

What I Truly Remember Is This Division of Labor

The most important thing about Tool Use is not the phrase "AI can call functions."

What's truly important is this division of labor:

The model is responsible for understanding and decision-making.

The runtime is responsible for execution and validation.

The tool is responsible for connecting to the external world.

The result goes back to the model, which continues to generate an answer the user can read.

So next time you see AI checking the weather, reading files, searching the web, or calling APIs, don't think of it as too mysterious.

It's not that it suddenly gained self-awareness, nor has the model truly broken through physical limitations.

Behind it is a very clear engineering process:

Describe the tools to the model, let the model select the tool, then let the program execute the tool, and finally hand the result back to the model.

The LLM is responsible for "figuring out what to do."

The Tool is responsible for "actually getting it done."