跪拜 Guibai
← Back to the summary

The 5,671-Token Greeting: Inside One Developer's Audit of Agent Token Costs

A "Hello" Used 5,671 Tokens: How I Chased Down the Hidden Costs of an Agent in Workrun

I initially assumed Workrun's conversation entry point should have a sufficiently capable main agent. Whatever the user said, it would first understand. Ordinary questions it answered itself; when tools were needed it called tools; when a Workflow was needed it handed off to a planning agent; after the plan was confirmed it handed off to an orchestration agent. Complexity stayed internal. The user only faced a single input box.

Until I sent:

Hello, how is your day going?

The reply was unremarkable, but Workrun's built-in usage stats showed 5,671 tokens.

That number made me uncomfortable. Not because the cost of one greeting was high, but because I couldn't explain it. The user had not asked for anything complex. Why would the system need to do so much preparation? If I couldn't account for this, then as I kept adding tools, Skills, and Agents, how could I ever judge whether the cost was reasonable?

My first suspicion was that the prompt was too long. Looking back, that suspicion was not wrong, but it wasn't enough either. What I actually changed later was the scope of what the model was asked to decide, and the way the system handed information to the model.

Screenshot 2026-11-14 14.00.21.png

What I Lacked Wasn't Another Total

Workrun already had monitoring. Besides the UI token stats, I had previously hooked up OpenTelemetry, using Rust's tracing and ADK's OTLP Layer to export runtime information.

The early core code was:

let filter = Targets::new()
    .with_target("workrun_lib", LevelFilter::INFO)
    .with_target("adk_", LevelFilter::INFO);

let layer =
    adk_telemetry::build_otlp_layer::<tracing_subscriber::Registry>(
        SERVICE_NAME,
        &endpoint,
    )?;

It was fine for checking execution flow. But the question I needed to answer had changed: not why a certain function was slow, but why a single model request needed such a large input.

A single user interaction can involve multiple model calls. Tool return values enter the next turn's context, and previous history gets read again. Knowing only the total usage, even drawing another chart, doesn't show which step is worth fixing.

I wanted to open one call directly and see the full messages, tool definitions, returned content, and why the next call happened. Langfuse's approach to LLM observability fit this debugging process much better.

A common misconception here: I did not replace OpenTelemetry with Langfuse. OpenTelemetry is responsible for collection and transport; Langfuse is one of the backends that receives and interprets that data. The original problem was that Workrun's LLM semantics and debugging views were insufficient, not that OpenTelemetry couldn't record them. Langfuse OpenTelemetry integration docs

image.png

I still reuse ADK's OTLP Layer, and additionally add a NativeAdapter that maps model calls to Generations, filling in role, session, and other info. The input used for debugging comes from the actual model request:

json!({
    "messages": request.contents.iter().map(content).collect::<Vec<_>>(),
    "tools": request.tools,
    "config": request.config
})

Message body collection is controlled by a switch. Tool calls record their parameters and results at the native execution boundary.

I emphasize "actual boundary" because I did fall into a trap later: after saving succeeded, the SDK call finished early. The business was done, but the outer observability layer didn't get a chance to record the tool return. I later moved recording to where the tool actually executes: record the result first, then end the flow.

This taught me that observability itself needs verification. Without it, you can optimize against an incomplete trace and easily change the wrong thing.

After integrating, I used an actual "Order Cancellation: Risk Routing and Regression Evaluation" Workflow to compare. On one side I looked at the definition and run results in Workrun; on the other I expanded the same run's trace in Langfuse, checking execution order, risk routing results, and the input/output, duration, and token usage of related calls via the collected spans.

"Order Cancellation: Risk Routing and Regression Evaluation" Workflow canvas and run results

Screenshot 2026-11-15 13.15.28.png

The same run's trace in Langfuse, expanding key routing and call details

Screenshot 2026-11-15 13.19.21.png

Here "restoring" means viewing an already recorded execution, not re-running the Workflow in Langfuse. The canvas shows how the flow is defined; the trace shows what actually executed this time. How much detail you can see depends on whether the corresponding execution boundary was fully collected.

This information let me keep asking: which path did an input actually take, why did a certain step call the model, how did a tool result enter subsequent context. The adjustments that followed also needed to be judged against actual results and regression evaluation, not just whether the total usage went down.

A "Capable Entry Point" Starts Getting Expensive

Workrun's original main agent simultaneously handled conversation, task processing, request scope judgment, and specialist delegation. The module comment in the code made this clear:

//! Main agent: conversation, direct work,
//! turn grants and specialist delegation.

Drawn by responsibility, it looked like this:

image.png

That doesn't mean every time it runs through the whole chain. The problem is that, to decide which path to take, the entry point gradually needs to know more and more.

At the time I didn't plan to add a routing agent. There was already a main agent making judgments. Wouldn't adding another layer overcomplicate a simple thing? I even hoped to keep the original structure and just shorten the prompt.

But the capability of ordinary tasks doesn't stop growing. Today it supports files, tomorrow it adds tools, the day after adds Skills. As long as the entry point still handles real tasks, its context will keep growing.

So in the end I split it. The reason wasn't "multi-agent is more advanced"; it's that the entry point's need to judge is relatively stable, while task capabilities keep changing. Those two kinds of change shouldn't stay bound together.

image.png

The Router only returns conversation, task, or workflow. It reads a limited message preview and state summary; it doesn't receive full tool results. The agent that actually handles the request still uses the original input.

The cost is clear: ordinary conversation now typically becomes two calls, one for routing and one for answering. Currently the roles still reuse the model the user selected; "lightweight" refers to context and responsibility, not to assigning a cheaper model.

So you can't just declare savings from the split. This choice first solves the boundary problem as capabilities grow; the cost still has to account for the added routing call, then look at the whole picture.

A Cheap Route Is Useless; the Expensive Mistake Is Going the Wrong Way

There was a real mistake in the split, and it left a deeper impression on me than the token numbers.

I typed:

Help me create a score judgment Workflow: input score, if greater than or equal to 60 output "Pass", otherwise output "Fail". The input score should be fillable on each run. First give me a plan, don't create or execute.

The model wrote a plan, but then ended with:

Which Workflow platform are you using? What exactly are the input node, condition node, output node called? I can't see your workspace from here.

It asked me, inside Workrun, what platform I was using.

Screenshot 2026-11-14 14.33.49.png

From the user's perspective, this path was already wrong: a planning request inside a product was being treated as a general consulting question.

"Plan first" was only a constraint to defer creation; it didn't turn the request into a normal chat. Later I made the Router explicitly handle this boundary:

"Plan first", "do not create yet" and "do not execute"
restrict later actions; they do not turn a Workflow
project request into general conversation.

After the fix, the same request went into the native planning flow and generated a plan card awaiting confirmation.

Screenshot 2026-11-14 14.38.34.png

I used the score judgment as a test not because the system only needs to handle such requests. It's simple enough that it lets me tell more easily: is this complexity coming from the user, or from the system itself?

This also added a precondition to later evaluation: first confirm the request went the right way and the result is usable, then compare how much it cost. A low usage count from going down the wrong path is meaningless.

Budget and Attribution Are a Separate Ledger

Langfuse let me ask why those tokens were spent. But team mode still needs to answer: who can call which models, how much total is allowed, and how much has each person spent?

For this I integrated LiteLLM Proxy.

When individuals configure models themselves, the call is straightforward:

image.png

Team models add a Server and a gateway:

image.png

Direct individual connections still exist. Team credentials stay on the server; budgets and rate limits are centralized in the gateway, no longer scattered across client ends. LiteLLM already provides Virtual Keys and related controls, so I didn't need to write another set inside Workrun. Virtual Key, budget and rate limits

Here's a point that's easy to confuse: sharing one Virtual Key among the team does not mean team members can only be aggregated into a single stat.

Workrun uses the End User field to distinguish members. The Server generates a stable identifier based on the authenticated user:

const endUser = `workrun:${settings.deploymentId}:${encodeURIComponent(userId)}`;

const request = {
  ...adaptJsonOutput(parameters, configuredAliases),
  user: endUser,
  metadata: {
    workrun_request_id: requestId,
    workrun_team_id: settings.teamId,
  },
};

deploymentId distinguishes deployments, userId distinguishes members. The end result looks like:

image.png

LiteLLM supports tracking End User consumption based on the user field in the request. Customer Usage docs

But this identity can't be arbitrarily filled by the client. The Server overwrites the client-declared field when writing. Tests also verified that even if two users submit forged identities, the requests still carry their real identifiers:

expect(sent.map((request) => request.user)).toEqual([
  "workrun:deployment-a:user-1",
  "workrun:deployment-a:user-2",
]);

Responses also write the same user, and additionally hash it for safety_identifier. The latter is another protocol field; it doesn't replace attribution.

The Virtual Key needs to be bound to a team at the gateway; workrun_team_id is just business metadata. Passing a user doesn't automatically set an individual quota. When you want per-member budgets, you still need to configure End User limits separately.

I want to keep this boundary simple: credentials handle team access, server-side authentication determines user identity, and request fields pass identity to the gateway. Don't add a whole Key lifecycle to the product just to track each member.

Screenshot 2026-11-14 14.44.12.png

The Gateway Unifies the Entry, but Doesn't Erase Model Differences

After integrating LiteLLM, switching to the then-configured DeepSeek immediately hit a 400:

The model gateway rejected the request.

error.type: invalid_request_error
error.code: upstream_error

Checking the actual request, the problem was structured output. Routing needs fixed JSON, and the original json_schema wasn't accepted by that model path. In the end I declared model aliases that need adaptation:

LITELLM_JSON_OBJECT_MODELS=deepseek-v4-flash

When that alias is hit, the format changes to json_object and the schema goes into the instruction. The core Chat Completions change:

return {
  ...parameters,
  response_format: { type: "json_object" },
  messages: [
    ...parameters.messages,
    {
      role: "system",
      content: instruction(format.json_schema.schema),
    },
  ],
};

Responses handle text.format and instructions separately. The code lives in apps/server/src/modules/llm/llm.output.ts.

JSON Mode does not guarantee the business fields conform to the schema. The application still keeps parsing and validation on its side. This adaptation only applies to the configured model paths; I won't infer capabilities for all DeepSeek deployments from this.

I didn't design a model capability probing framework. What I needed was a compatible entry that could be explained and tested. Making it more general wouldn't have helped this debugging.

Prompts Got Shorter, but the Whole Turn Could Still Cost More

In one turn, before the user even confirmed the plan, the planning stage used 31,732 tokens.

Screenshot 2026-11-14 14.46.55.png

The next few turns went in circles: delete prompt text, shorten reference content, test again. Sometimes it went down, the next turn it was high again.

The easiest mistake to make in this period is treating "shorter input explanation" as "lower task cost."

When the model lacks configuration knowledge, it goes and queries tools. After a tool query, the result enters context, and the next call happens. The explanation you saved may be outweighed by the extra round of requests you added.

So later I focused on where information appears, not simply on giving less.

During planning, first give an overview of node capabilities, enough to judge which execution approaches are feasible. During orchestration, with a confirmed plan, the program can provide relevant contracts in advance. context_contracts collects Start, End, plan-suggestion nodes, and the nodes the existing definition uses, then reads their contracts:

reference(&json!({
    "topic": "nodes",
    "node_types": kinds,
    "detail": "summary"
}))

Node contracts keep the full dataSchema, semantics, and branch handles. The suggestions in the plan are not hard constraints; other nodes can still be queried when needed.

I also fixed a very specific issue: when requesting detail=summary, the examples for specified nodes could still return full text. Now the summary branch returns only indexes:

if kinds.is_empty() || args["detail"] == "summary" {
    json!({
        "id": example["id"],
        "node_type": example["node_type"],
        "summary": example["summary"]
    })
}

These adjustments don't promise to "make the model smarter"; they just make it search less for things it already knows and read less content it doesn't currently need.

Screenshot 2026-11-14 15.06.17.png

What Really Doesn't Need to Ask the Model Anymore

As optimization went further, I started checking the reverse: do some calls even need to exist?

When the user clicks the confirm button, the program already knows the plan ID and version. Making a general intent model explain "confirm version 1" adds no information, but it adds cost and room for misinterpretation. Now the native state directly restores the corresponding flow.

Saving is the same. The orchestration agent summarizes after saving, then the upper layer receives the result and summarizes again twice, each time carrying context. For a create-only, no-execution request, the native tool's successful return is enough to end that phase.

The actual stop condition still checks save success and execution scope:

if result.is_ok()
    && agent == "workflow_author"
    && name == "save_workflow_draft"
    && state.saved.is_some()
    && !state.registry.execution_allowed
{
    // Record the native receipt, end this call.
}

However, the first time I removed the summary, the screen only showed the card. The normal "created successfully" reply I wanted was also gone.

Screenshot 2026-11-14 14.52.41.png

Screenshot 2026-11-14 14.53.19.png

In the end I added a native-receipt-based message outside the card, clearly stating save, validation, publish, and execution status, without adding a model call or inserting a text node into the Workflow.

This kind of omission can't be judged by the call tree alone. When you remove a step, you still have to catch the product feedback it was providing.

Screenshot 2026-11-14 15.09.33.png

Screenshot 2026-11-14 15.09.47.png

Another more awkward problem: one reply and the card both showed save succeeded, but opening it returned Not Found, and the list didn't show it either.

Screenshot 2026-11-14 14.55.19.png

During debugging I was tempted to change team drafts directly to server-side storage, but later corrected myself. Workrun's original convention is that drafts live locally and only publish goes to the Server. I couldn't change that product semantics just to fix a read problem. In the end I kept local drafts, and the team list merges local drafts with server-published content.

This made me more careful about the word "done." The model generated the config, the tool saved successfully, static validation passed, the user can open it, and the actual run is correct: these are different checks. In this round we only created, didn't execute, so we shouldn't write static validation as successful run validation.

Agree on What "Orchestration Success" Means Before Discussing Savings

Earlier there was a question not yet clarified: what exactly are we measuring when we talk about an orchestration conversation? If the only standard is fewer tokens, having the model directly output a piece of JSON that looks like a Workflow might be cheapest, but I wouldn't dare use it.

Workrun's nodes are defined by the product itself. The model knows what a conditional branch is, but that doesn't mean it knows what Workrun's branch handles are called, how fields are filled, or how one node's result passes to the next. These can't be patched from general knowledge.

So now I use the following set of criteria to check an orchestration. It may not be the final answer, but it gives optimization a common premise.

First, the model must orchestrate with evidence. It needs to understand Workrun's current document structure, connection and state rules, and what existing nodes can do. After choosing a certain node type, it must read that node's actual usage and configuration contract before generating config. Fields, input/output, and execution limits should all be traceable to the platform's own documentation.

The "reading" here doesn't necessarily mean it must call a tool once. If the full contract is already provided by native context, the model can use it directly; if not, it must query. You can't substitute a node synopsis for the config doc, and you can't assume the model can configure Workrun just because it has seen other Workflow products.

Second, the artifact must match the user's need. If the user requires an input that can be filled each run, it can't be hardcoded into the definition. If deterministic judgment is required, there needs to be explicit comparison and output logic. Nodes being connectable is just structural validity; you also have to check where inputs come from, who produces the result, and how it finally returns. Features that weren't asked for shouldn't be added just to look complete.

Finally, validate only to the point where you've completed. If only planning was requested, produce an auditable plan and don't create. After creation is confirmed, check config validation, actual save, and whether it can be reopened. After run is authorized, verify behavior with inputs and expected outputs. Static validation passing doesn't equal run success; if you haven't executed, clearly say it hasn't been run-verified. The user should also be able to tell from the conversation what the current state is, not have to guess what a card means.

In a complete interaction, the evidence I expect to see looks roughly like:

Understand the request and operation scope
    ↓
Read Workrun orchestration rules and node capabilities
    ↓
Form a plan, wait for user confirmation
    ↓
Read usage and full config contract for selected nodes
    ↓
Generate config, validate, save
    ↓
Confirm artifact is accessible, state save and execution status
    ↓
If run is authorized, check actual results

This is a checklist; it doesn't require each box to correspond to a model or tool call. Relevant documentation can be provided in advance, and the same valid contract can be reused, but you can't skip these grounds.

With this standard, when I open Langfuse, I don't treat every reference query as waste. Reading the config doc for a selected node the first time is necessary cost. Reading the same manual repeatedly when the full contract is already in context is what deserves investigation. Or deleting key semantics to compress input, only to have the model guess fields, fail validation, and then fix it, isn't a successful optimization either.

So the comparison order becomes: first check if this standard is met, then compare usage, duration, and repair count under the same model, same task, and same phase. A request that's a little cheaper but easier to misconfigure and needs constant user correction isn't an effective improvement I'd accept.

Screenshot 2026-11-14 15.41.46.png

Screenshot 2026-11-14 15.42.30.png

Why I Finally Stopped

I didn't keep chasing down every higher trace later.

The earlier stretch was close to a loop: new data high → change, next data low → think it worked, then high again → overturn. I even switched models in the middle, so conditions weren't fixed, and many judgments couldn't be compared.

Later I fixed inputs and conditions, and evaluated planning and creation separately. Changes that didn't show improvement kept the baseline. I also stopped taking the initial greeting's 5,671 tokens and comparing them to the usage of some full creation to compute a "savings ratio" — they aren't the same thing.

If I had to summarize what changed me most, it's that I started treating tokens as a clue about system behavior. High consumption can come from necessary work, or it can indicate responsibilities are in the wrong place, or that one thing is being repeatedly confirmed and re-explained.

Langfuse let me check those judgments, and LiteLLM gave the team a unified management entry for calls. Neither of them decided for me how the agent should work.

With that standard in place, asking "why does this step exist" has meaning. The first time the model reads a node's usage, it's filling in orchestration evidence. Reading an existing contract repeatedly, or re-explaining a confirmed state, is likely duplicate work. I want to reduce the latter while keeping the former, the steps that make results trustworthy.

Going forward, when I add capabilities to Workrun, I'll ask these questions first. That's better than adding features and then auditing the ledger starting from a "hello."

Comments

Top 1 of 2 from juejin.cn, machine-translated. The original thread is authoritative.

组合搭配函数

The example of the grading logic taking the wrong path is more worth remembering than the 5,671 number — "give plan first" was treated as a general consultation, and the model asked back which platform you use, which shows that when routing goes wrong once, all the tokens saved are fake savings. I agree with the boundary 'Plan first, do not create yet and do not execute restrict later actions; they do not turn a Workflow request into a general chat', but I want to ask: is this boundary currently only written in the Router's prompt, or has it been turned into a regression test case? Boundaries in prompts are easily diluted by the next round of requirements; regression test cases are what guarantee it won't quietly drift off course.

梦想很大很大

Currently there is automated test coverage, but we still need to improve the evaluation system to keep checking this kind of routing boundary.