A 94.5% Cache Hit Rate Cut This Coding Agent's Costs by 85%
Coding agents are input-heavy by design, and a high cache hit rate is the single biggest lever for controlling cost. This breakdown shows that a 94.5% hit rate—achieved through deliberate, cache-aware engineering rather than luck—can reduce input costs to one-seventh of the uncached baseline, turning what would be a prohibitive bill into pocket change.
A one-hour stress test of the open-source coding agent DeepSeeker-Code on the deepseek-v4-flash model consumed 25.68 million tokens at a total cost of ¥2.26. The output was a negligible 195,000 tokens; the agent's cost structure is dominated by repeatedly re-sending the full context—system prompts, tool definitions, and conversation history—which made input 130× larger than output. A 94.5% cache hit rate on those input tokens slashed the input bill by roughly 85%, because DeepSeek's context caching charges about one-tenth the standard price for repeated prefixes. That hit rate is not accidental. It is preserved through strict engineering constraints: the system prompt is immutable and always occupies the first message position, compression is deferred when the cache is healthy and triggered early when it is already broken, and the message structure keeps fixed slots for summaries and tool definitions. These choices trade away flexibility and lock the agent into DeepSeek's specific caching behavior, but for a high-frequency local agent the cost savings make the coupling worthwhile.
Agent economics invert the usual LLM cost intuition: output is cheap and negligible, while the repeated re-transmission of a growing context dominates the bill.
Cache hit rate is not a passive property of the model provider; it is an engineering outcome that must be actively preserved through strict constraints on prompt structure and compression timing.
The trade-off between flexibility and cost is stark—every structural change to the system prompt risks shattering the cache and multiplying the bill, so stability becomes a hard requirement rather than a nice-to-have.
Tying compression logic to cache health is a pragmatic heuristic that turns a destructive operation into a cost-aware decision, but it also deepens the agent's coupling to a specific model's caching implementation.