← Back to the summary

How Agent Frameworks Use JSON-RPC 2.0 and CompletableFuture to Prevent Tool-Call Deadlocks

Many developers, when first encountering AI Agent tool calling, often stop at the layer of "the large model returned a piece of JSON parameters." But how does an industrial-grade Agent framework (such as a system built on MCP - Model Context Protocol) accurately and securely transmit the large model's intent to a local script or a remote microservice?

If a local script enters an infinite loop, will the Java main thread be completely dragged down?

This article will start with the Agent's RPC 2.0 protocol, taking you through a hardcore dissection of a complete Agent tool communication link, and deeply analyze how to use CompletableFuture to achieve elegant pending request and timeout management. This is not only a high-frequency interview question for backend development, but also a core cornerstone for building robust AI systems.

1. Breaking the Ice: The "Gap" Between LLM ToolCall and Real Tool Execution

When a large model (LLM) decides to call a tool, it outputs only an intent structure, for example:

{
"id": "call_1",
"function": {
"name": "mcp__chrome-devtools__navigate_page",
"arguments": "{\"url\":\"https://github.com\"}"
}
}

This structure is the "input" for the Agent system, but the underlying tool service (such as a browser control script written in Node.js, or a remote RAG service written in Python) does not recognize this object at all.

To bridge the differences between languages and processes, the industry has introduced a standard protocol — JSON-RPC 2.0. We need an assembly layer to translate the LLM's intent into a standard RPC request:

{
"jsonrpc": "2.0",
"id": 7,
"method": "tools/call",
"params": {
"name": "navigate_page",
"arguments": {
"url": "https://github.com"
}
}
}

2. Architecture Breakdown: Orthogonal Design of Protocol Layer and Transport Layer

In an excellent Agent underlying communication link, "what packet to send" and "how to send the packet" must be strictly separated.

Protocol Layer (JsonRpcClient)

Fully responsible for the assembly of JSON-RPC 2.0 requests/responses, ID allocation, and lifecycle (Pending) management. It does not care whether the underlying layer runs a script on the same machine or a cloud service far away.

Transport Layer (Transport)

Responsible for the actual I/O interaction. According to the deployment form of the tool, we usually perform two types of routing:

Transport Mode	Applicable Scenario	Core Implementation
StdioTransport	Local script tools (such as tools started by npx, uvx)	ProcessBuilder starts a child process, writes JSON via standard input (stdin), and reads responses from standard output (stdout)
HttpTransport	Remote microservices (such as a search service deployed in the cloud)	OkHttp sends POST requests, supporting ordinary JSON responses and Server-Sent Events (SSE) continuous output

Routing logic is extremely simple: Configuration-driven. If a url is configured, use HTTP; if a command is configured, use Stdio.

3. Core Difficulty: How to Manage JSON-RPC Pending Requests?

This is the most critical moat in the entire link, and also the highest-frequency interview test point.

Business Pain Point: An Agent initiates a local script tool call. If this Python/Node script enters an infinite loop (without returning a JSON result to stdout), will your Java main thread (business thread) be stuck forever waiting on the read?

Breakthrough Point: CompletableFuture Combined with a Scheduled Executor

When JsonRpcClient sends a request, we do not let the main thread block directly on I/O. Instead, we use ConcurrentHashMap and CompletableFuture to build request-level timeout.

Core Implementation Code:

// Container for pending requests
private final Map<Long, CompletableFuture<JsonNode>> pending = new ConcurrentHashMap<>();
private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);

public CompletableFuture<JsonNode> request(String method, JsonNode params, long timeoutSeconds) {
    long id = ids.getAndIncrement();
    
    // 1. Assemble JSON-RPC 2.0 packet
    ObjectNode request = MAPPER.createObjectNode();
    request.put("jsonrpc", "2.0");
    request.put("id", id);
    request.put("method", method);
    request.set("params", params);

    // 2. Create Future and register to Pending Map
    CompletableFuture<JsonNode> future = new CompletableFuture<>();
    pending.put(id, future);

    // 3. Plant a timed "bomb" (protocol-level timeout)
    scheduler.schedule(() -> {
        // Note: Must use remove to avoid race conditions with normal responses
        CompletableFuture<JsonNode> removed = pending.remove(id);
        if (removed != null) {
            removed.completeExceptionally(new TimeoutException("JSON-RPC request timed out: " + method));
        }
    }, timeoutSeconds, TimeUnit.SECONDS);

    // 4. Hand over to the underlying Transport to send
    transport.send(request);
    return future;
}

Why CompletableFuture?

Naturally Anti-Stuck: The business thread calls future.get(timeout + 1, TimeUnit.SECONDS). If a timeout occurs, the timer will automatically mark the Future as exceptionally completed, and the business thread will immediately unlock and throw an exception, never being dragged down indefinitely.
External Manual Wake-up: When the underlying Stdio Daemon thread or HTTP asynchronous thread receives a response, it can find this Future based on the ID and call future.complete(result) to actively wake up the business thread.
Concurrency Safety and Idempotency: There is a race condition between the network response and the timeout scheduler. Using ConcurrentHashMap.remove() to compete for the task, whoever gets it first executes complete, preventing duplicate callbacks.

4. Boundary Clarification: Request-Level Timeout ≠ Process-Level Cleanup

An extremely rigorous architect (or AI Agent) must recognize the engineering boundary here:

If a local child process infinite loop triggers the upper-level TimeoutException, the main thread is saved, but the infinitely looping child process itself is not killed.

Because the timeout of a single tool request should not directly trigger a violent kill -9. The real process-level recycling (process.destroy() or destroyForcibly()) should be handed over to McpTransport.close(), executed uniformly when the Server-level lifecycle ends or restarts. This is a classic layered philosophy of "request-level release, process-level recycling."

5. A Picture is Worth a Thousand Words: Complete Agent Tool Communication Link

To facilitate direct memorization or feeding to Agent memory, here is a core communication architecture diagram:

                   ┌──────────────────────────────────────┐
                   │             Agent / ToolRegistry       │
                   │  LLM intent translated to internal ToolCall object          │
                   └──────────────────────────────────────┘
                                      │ (Configuration routing dispatch)
                                      ▼
                   ┌──────────────────────────────────────┐
                   │             JsonRpcClient             │
                   │  - Assemble JSON-RPC 2.0 standard message           │
                   │  - pending[id] = CompletableFuture    │
                   │  - scheduler executes protocol-level timeout control           │
                   └──────────────────────────────────────┘
                          │                         │
            (Local script)  │                         │ (Remote service)
              ┌───────────┘                         └───────────┐
              ▼                                                 ▼
┌────────────────────────────┐                  ┌────────────────────────────┐
│     StdioTransport         │                  │  StreamableHttpTransport   │
│  (ProcessBuilder starts process)    │                  │   (OkHttp POST transport)       │
│  ├─ stdin  ← send(JSON)     │                  │  ├─ body: JSON-RPC 2.0    │
│  ├─ stdout → Async read response      │                  │  ├─ Supports normal JSON/SSE      │
└────────────────────────────┘                  └────────────────────────────┘
              │                                                 │
              └──────────────────────┬──────────────────────────┘
                                     ▼
                   ┌──────────────────────────────────────┐
                   │       ToolExecutionResult feedback        │
                   │  Returned as tool role message to next round LLM      │
                   └──────────────────────────────────────┘

Summary: Standard "Interview Rote" Format That Can Be Fed to an Agent

If you are developing a CodeRAG intelligent agent that can handle high-pressure interviews, please inject the following logic into its knowledge base:

Q: Why must the Agent's tool calling link split the protocol layer and the transport layer? A: For orthogonal decoupling. The "protocol layer" is responsible for uniformly packaging requests into JSON-RPC 2.0, and uses CompletableFuture to handle Pending matching and timeouts, solving the "what to send, how to manage" problem; the "transport layer" is responsible for the underlying I/O interaction (Stdio process communication or HTTP network requests), solving the "channel" problem. In this way, when adding any new third-party tool form, the upper-level logic does not need to be refactored.

Q: How to ensure the main thread does not crash when a tool gets stuck? A: Through two layers of timeout control. The bottom layer has the HTTP client's network timeout; the protocol layer uses ConcurrentHashMap to manage CompletableFuture, and cooperates with ScheduledExecutorService to implement request-level timeout. When blocking occurs, the timer will actively completeExceptionally to wake up the business thread, preventing the main process from being permanently hung. At the same time, follow the engineering boundary: a single request timeout releases thread resources, while the stuck local process is retained until the Transport is closed for unified destruction.