跪拜 Guibai
← All articles
Frontend · Backend · Artificial Intelligence

The 5,671-Token Greeting: Inside One Developer's Audit of Agent Token Costs

By 梦想很大很大 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Agent cost overruns are usually invisible: total token counts hide how many calls a turn triggers and how much context each call carries. This audit shows that tracing individual calls, splitting responsibilities, and agreeing on a success standard before optimizing are what actually keep agent costs predictable and debuggable.

Summary

Workrun's main agent was designed as a single capable entry point: understand the user, answer, call tools, route to plans, and orchestrate. That design broke down in practice when a plain greeting burned 5,671 tokens and nobody could explain why. The author traced the cause by giving up on aggregate stats and instrumenting individual model calls through Langfuse, reusing the existing OpenTelemetry layer. Observability revealed a bloated context that carried responsibilities far beyond the immediate question.

The fix was structural, not cosmetic. The main agent was split into a lightweight router that only decides between conversation, task, and workflow, while the agent that actually does the work retains the full input. Splitting introduced a second call per ordinary turn, so savings aren't automatic. More importantly, the author found that shorter prompts can backfire: when the model lacks configuration knowledge, it queries tools, pulls results into context, and triggers extra rounds, so total consumption can rise even with a trimmed prompt. The right move was providing node contracts and summaries at the right stages instead of simply withholding information.

Cost control also required knowing what "success" means. The author defines three checks before comparing token counts: orchestration must be grounded in the platform's actual node contracts, the artifact must match the user's stated requirements, and validation must only claim what was actually executed. Only after those are satisfied does it make sense to compare usage, duration, and repair counts across model calls.

Takeaways
— A single greeting cost 5,671 tokens because the main agent's context bundled conversation, tool handling, scope judgment, and specialist delegation.
— OpenTelemetry exports the trace, but Langfuse provides the LLM-level view that lets you inspect individual calls, messages, tool definitions, and why the next call happens.
— Observability needs its own verification: recording at the wrong boundary can miss tool returns, so traces must be recorded where the tool actually executes.
— Splitting the main agent into a router that returns only conversation, task, or workflow adds a second call per ordinary turn; savings are not automatic.
— A router that misroutes is more expensive than a slightly larger context: the author's model once asked which Workflow platform it was running on, so the router needed an explicit rule that 'plan first' requests are still Workflow work.
— Shorter prompts can increase total token usage if the model compensates by querying tools and pulling results into context, so the fix is providing relevant contracts and summaries at the right stage, not simply giving less.
— LiteLLM's Virtual Keys and End User fields allow per-member budget tracking even when the team shares one key, but the identity must be set by the server, not the client.
— To make DeepSeek work, the gateway needed `json_object` mode with schema pushed into a system instruction instead of `json_schema`, and the change was scoped only to configured model paths.
— Remove model calls only if the product feedback they provided is otherwise handled: after dropping post-save summaries, the UI had to add a native-receipt-based status message.
— An orchestration evaluation standard must be met before comparing token usage: grounded node contracts, artifact that matches user requirements, and validation limited to what was actually executed.
— Token counts are a clue, not a score; high usage may be necessary work or it may indicate misplaced responsibilities or repeated confirmation.
— Optimization loops fail without fixed inputs and conditions; comparing a greeting to a full creation to compute a 'savings ratio' is meaningless.
Conclusions

Total token usage is a poor health metric for agents because it conflates the number of calls with the size of each call; per-call tracing exposes the real cost driver.

A router that always requires two calls can still be cheaper than one call with huge context, but only if the router never misroutes; a wrong path is the expensive failure.

Shorter prompts are a trap when they push the model into tool queries; the more effective pattern is to make existing knowledge readily available at the moment it's needed.

Discipline in defining what counts as success is what makes token optimization a real engineering process instead of a game of whack-a-mole.

The author's decision to avoid a generic model capability detection framework shows that scoping fixes to actual failures is more valuable than speculative abstraction.

Concepts & terms
Router agent
A lightweight agent that receives only a message preview and state summary and returns a category like conversation, task, or workflow. It narrows the decision scope without loading the full context, so the specialized agent that handles the actual task gets a cleaner input.
Token accounting
Tracking how many tokens each model call consumes in an agent system, including the context that accumulates across multiple calls in a single user turn. The article argues aggregate totals hide which step is worth optimizing, so per-call tracing is required.
End User field
A field in an LLM gateway request that identifies the human user for consumption tracking. When set server-side and protected from client forgery, it lets a shared Virtual Key still attribute usage to individual team members.
From the discussion
Featured comments
组合搭配函数

The example of the grading logic taking the wrong path is more worth remembering than the 5,671 number — "give plan first" was treated as a general consultation, and the model asked back which platform you use, which shows that when routing goes wrong once, all the tokens saved are fake savings. I agree with the boundary 'Plan first, do not create yet and do not execute restrict later actions; they do not turn a Workflow request into a general chat', but I want to ask: is this boundary currently only written in the Router's prompt, or has it been turned into a regression test case? Boundaries in prompts are easily diluted by the next round of requirements; regression test cases are what guarantee it won't quietly drift off course.

梦想很大很大

Currently there is automated test coverage, but we still need to improve the evaluation system to keep checking this kind of routing boundary.

See top comments, translated →
Source: juejin.cn ↗ Google Translate ↗ Backup ↗