Running Production Agents Steadily: SSE Streaming, HITL Pause/Resume, and the State That Holds It Together
Preface: The closed beta has come to an end, and I'm starting a systematic review: how a production Agent actually runs. Pitfalls, lessons, and implementation — I'll try to cover it in a few articles. This is Part 5 of the series:
Once an Agent goes into production, the problems are often no longer "can the model answer" but: a task runs for a long time with no feedback to the frontend; the user clicks confirm, but the backend doesn't know which step to continue from; after a page disconnects, progress is lost, and it might even re-execute a dangerous operation.
I hope this article gives some reference to those working on Agent production deployments, and feel free to share your experiences and questions in the comments.
User says: "Restart this abnormal instance." Partway through the investigation, they add: "Keep investigating it."
For ordinary chat, these might just be two natural language sentences; but for a production Agent, the system also needs to know which instance "it" refers to, which step the current task is at, and whether user confirmation is needed before restarting.
This article covers only two things: how a single request continuously pushes progress to the frontend, and after user confirmation, how to continue execution from the correct breakpoint.
A single-round chat API isn't complicated. Once you actually integrate it into a business, the following three types of problems typically appear quickly:
- User says "keep investigating it" and the system doesn't know who "it" is;
- The Agent runs for a long time, and the frontend has no feedback for a long while;
- Deletion, publishing, and transfers must be confirmed by a human, and after confirmation, execution must continue from where it was interrupted.
These three types of problems correspond to three core capabilities: multi-turn state management, SSE streaming communication, and HITL (Human in the Loop). We'll continue using the fictional case OpsPilot (an enterprise ops assistant), with TypeScript as the tech stack; the frontend can use Vue 3 with fetch to read the stream.
Different scenarios have different emphases:
- Knowledge Q&A: Focus is on incremental text push, first-token response speed, and result recovery after disconnection.
- Long-running tasks: Focus is on task state persistence, progress querying, and decoupling frontend-backend connections.
- Ops, publishing, and approvals: Focus is on human confirmation, permission verification, concurrency control, and reliable execution of side effects.
If you're just building a chat demo, many of these details can be ignored; but if you're connecting an Agent into a real business system, these designs will be faced sooner or later.
A run should have its own state
Don't just store everything as a single chat text string. At minimum, distinguish three types of data (can be three tables, or three separate records), because their lifecycles and purposes differ:
- Conversation: one item in the chat list on the left, e.g. "troubleshoot - payment-api". A person can have many conversations.
- Message: each utterance in a conversation: what the user said, what the assistant replied, tool results — each counts as one.
- Run: when a user clicks send, the backend opens one task for that message (runs the model, calls tools, waits for human confirmation). A single conversation can have many Runs.
type Conversation = { // a chat conversation (an item in the chat list)
id: string // conversation id
userId: string // which user it belongs to
createdAt: string // creation time
}
type Message = { // a message within a conversation
id: string // message id
conversationId: string // which conversation it belongs to
role: 'user' | 'assistant' | 'tool' // who said it
content: string // body (display / write to db)
createdAt: string // creation time
}
type Run = { // one backend run corresponding to one user click send
id: string // Run id
conversationId: string // which conversation it falls in
userMessageId: string // which user message triggered it
status: // how far this run has progressed
| 'running' // in progress
| 'interrupted' // waiting for human / supplementing params
| 'completed' // normal end
| 'failed' // failed
| 'cancelled' // cancelled
checkpoint?: unknown // minimal state needed to continue after interruption
createdAt: string // creation time
updatedAt: string // last status change time
}
One user request corresponds to one Run. After user confirmation, you should resume this run, not start a new chat round and let the model guess what happened before. This preserves task context and reduces the risk of repeated tool calls.
Multi-turn context: make the model understand "it" but don't stuff the entire history in
Users often say "continue investigating it" or "the one just now". If you hand the model the complete chat record every time, it costs more and old information may interfere with the current task.
You can split context into three layers:
- Confirmed business facts: structured
state, e.g. service name, environment, instance ID. Only update after tool results are validated or the user explicitly confirms. - Earlier conversation summary: compressed into a summary to control length.
- Recent turns verbatim: keep what the user and assistant said most recently so the model can understand references like "it" or "the one just now".
Among these, state should come from trusted business data sources as much as possible; the summary is auxiliary information to compress history, and just because something is in the summary doesn't mean it's a confirmed fact.
// 1) confirmed facts (attached to the conversation)
const state = { service: 'payment-api', env: 'prod' as const }
// 2) summary of earlier conversation
const earlierSummary = 'User is troubleshooting a production error; service name and environment are confirmed.'
// 3) recent verbatim turns (including the current "continue investigating it…")
const recentMessages = [
{ role: 'user', content: 'Look at the recent errors of payment-api' },
{ role: 'assistant', content: 'There have been 12 5xx errors in the last 15 minutes in production.' },
{ role: 'user', content: 'Continue investigating its recent error logs' },
]
// Assemble: 1+2 → system; 3 → message list
const modelInput = [
{
role: 'system',
content:
`Known facts: ${JSON.stringify(state)}\n` +
`Earlier summary: ${earlierSummary}\n` +
`Use the recent verbatim turns to understand who "it / the one just now" refers to; don't guess by keywords.`,
},
...recentMessages,
]
This is just an example of context assembly. A real project also needs to handle message roles, prompt boundaries, permission filtering, context length, and sensitive information masking.
First look at a complete chain (SSE + HITL)
Taking OpsPilot as an example, the user says: "Restart this abnormal instance."
This kind of operation should not be executed immediately just because the model proposes a tool call. It should be split into two stages: first query and clarify the operation target, then wait for user confirmation; only after confirmation passes, actually execute the restart.
Stage 1: send message, watch progress, pop up confirmation
- Frontend sends the sentence to the conversation service, and simultaneously opens an SSE to receive progress (started, checking, finished checking…).
- Backend first runs read-only steps (e.g. locate which instance), and these progress updates are pushed via SSE to the page.
- Model proposes to call the "restart" tool. At this point, do not execute the restart. Instead, send an
interruptevent: the page pops up a confirmation (risk explanation, who will be restarted). - This Run is marked as "waiting for human"; this round of streaming can end. On the page it's a confirmation, not a red error.
Stage 2: user clicks confirm, continue from breakpoint
- User clicks "Confirm". The frontend calls the resume API (carrying the interrupt id, approve/reject, and the same request key to prevent double-click). It is not sending a whole new chat.
- Server side first confirms: this interrupt is still valid, the current user still has permission, the instance is still in that state; only then actually restart.
- Execution progress is pushed again via SSE (tool start/end, final answer), until completion.
sequenceDiagram
participant U as User
participant V as Frontend
participant S as Conversation service
participant R as Runtime
U->>V: Restart this abnormal instance
V->>S: send message and subscribe SSE
S->>R: start this Run
S-->>V: progress: start / querying…
R-->>S: need confirmation (don't restart yet)
S-->>V: pop up confirmation
U->>V: click confirm
V->>S: call resume API
S->>R: re-verify permissions then execute restart
S-->>V: progress: restarting / final answer
Two things to remember here:
- Waiting for user confirmation is expressed with
interrupt; this is a normal business state, not an execution failure. - After user confirmation, go through the resume API and continue the original Run; don't treat it as a new chat round.
Also, a successful event push does not mean the state has been persisted. Interrupt records, checkpoints, and Run status should be reliably persisted before notifying the frontend. Otherwise, if the process crashes after sending the event, the page might show a confirmation card but the backend has no recoverable breakpoint.
SSE vs WebSocket: not everything complex is better
SSE (Server-Sent Events) is based on HTTP and lets the server continuously push events to the client.
For the scenario "user initiates a request, server continuously returns progress", SSE is usually simpler than WebSocket; if the business needs high-frequency bidirectional communication or binary data transfer, then consider WebSocket.
| Comparison dimension | SSE | WebSocket |
|---|---|---|
| Communication direction | Primarily server pushes to client | Bidirectional |
| Integration complexity | Generally lower, based on HTTP | Requires managing bidirectional message protocol and connection lifecycle |
| Event format | Has a standard event stream format | Message format defined by application |
| Auto reconnect | Native EventSource support |
Usually requires implementing reconnect strategy yourself |
| Missed event resend | Requires server support for event replay | Also needs application-layer design |
| More suitable for | Agent text output, tool progress, task status | Real-time collaboration, high-frequency bidirectional interaction, bidirectional control |
The most important thing here is not "which has higher performance" but whether the communication pattern fits the business need.
If the Agent mainly outputs text, tool status, and confirmation events to the page, SSE is often enough. If the frontend needs to frequently send control messages to the server, or the system itself is high-frequency bidirectional real-time interaction, then consider WebSocket.
Performance also can't be judged only by protocol name. Actual behavior is affected by connection count, event size, send frequency, proxy buffering, server resource usage, and message handling logic. Don't jump to conclusions that one scheme is definitely faster without load testing data.
Event protocol: let the frontend clearly know what happened
Some orchestration frameworks push not "the step that just happened" but "the complete state so far". For example, each update re-sends the entire step list.
If you directly convert all these states into frontend events, the same tool step might be displayed repeatedly. So you need to compare before and after states, or deduplicate by stable step ID, call ID, and only convert truly new steps into business events like tool_started.
It's recommended to define a set of stable business events first:
run_started: task started.text_delta: assistant text increment.tool_started: tool begins execution.tool_finished: tool execution ends.interrupt: task paused, waiting for user confirmation or additional information.done: task ends normally.error: task execution failed.
Protocol notes:
- Encoding is UTF-8, response type is
text/event-stream; charset=utf-8. - Each business event should carry a
sequencethat increments within this Run, for deduplication and later progress recovery. sequenceis a field in the business JSON, not the SSE protocol's ownid:field.- Text, parameter summaries, and error messages in events should be masked as needed; don't expose access tokens, internal stack traces, or full tool results.
import type { ServerResponse } from 'node:http'
type StreamEvent =
| {
type: 'run_started'
runId: string
sequence: number // within this Run 1,2,3…; prevents duplicate rendering, and can be used later to pull missed progress by number
}
| {
type: 'text_delta'
runId: string
sequence: number
delta: string // incremental text, not full overwrite
}
| {
type: 'tool_started'
runId: string
sequence: number
callId: string // paired with tool_finished
tool: string // tool name, for step bar display
}
| {
type: 'tool_finished'
runId: string
sequence: number
callId: string
tool: string
status: 'success' | 'error'
summary?: string // short summary for frontend, not raw large result
}
| {
type: 'interrupt'
runId: string
sequence: number
interruptId: string // returned as-is on resume
payload: unknown // confirmation: risk explanation, parameter summary, etc. (masked)
}
| {
type: 'done'
runId: string
sequence: number
messageId: string // assistant message id that has been written to database
}
| {
type: 'error'
runId: string
sequence: number
code: string
message: string // displayable to user; no stack trace
}
// Snippet: called in a route where a Run has been created and is about to stream (res is Node/Express response, not fetch Response)
function openSse(res: ServerResponse) {
res.setHeader('Content-Type', 'text/event-stream; charset=utf-8')
res.setHeader('Cache-Control', 'no-cache, no-transform')
res.setHeader('X-Accel-Buffering', 'no') // only nginx; can omit if not used
res.flushHeaders()
}
function sendEvent(res: ServerResponse, event: StreamEvent) {
// one SSE message: event line + data line + blank line ends
res.write(`event: ${event.type}\n`)
res.write(`data: ${JSON.stringify(event)}\n\n`)
}
// When needing human input: first write interrupt and Run=interrupted to database, then push event (schematic)
function emitInterrupt(
res: ServerResponse,
runId: string,
sequence: number,
interrupt: { id: string; payload: unknown }
) {
sendEvent(res, {
type: 'interrupt',
runId,
sequence,
interruptId: interrupt.id,
payload: interrupt.payload
})
// then you can res.end() to end this round; user confirmation goes through POST resume, no need to occupy the same long connection
}
Event types should express clear business meaning. Don't make the frontend parse the model's natural language output and guess "is a tool being called right now". Model text is for the user, business events are for the program — the two have different responsibilities.
One more thing: if you switch to browser-native EventSource, named events with an event: field don't go into onmessage; you need to listen via addEventListener('event name', ...). When using fetch to parse SSE yourself, you can dispatch directly from the type field of the JSON — don't confuse the two approaches.
How the frontend reads the stream and handles interrupt
Native EventSource can only send GET requests, and can't carry a request body or arbitrary custom headers like a normal fetch. So if the chat API needs a POST body, you can use fetch to read the response stream.
Both client-side approaches have trade-offs:
| Approach | Advantages | Things to note |
|---|---|---|
EventSource |
Browser handles SSE parsing; supports auto reconnect and Last-Event-ID |
Request method and headers are limited; server still needs to implement event replay |
fetch + ReadableStream |
Supports POST body, custom headers, suitable for chat APIs | Need to parse events, handle exceptions, and design reconnect yourself |
First POST to create Run, then subscribe with EventSource |
Clear separation of task creation and progress subscription | One extra request; need to design Run ID, subscription auth, and recovery protocol |
If you adopt "first POST to create Run, then subscribe SSE", it's usually easier to separate the task lifecycle from the connection lifecycle. If you want simplicity for the first version, you can directly use POST + fetch, but be clear that it doesn't have built-in disconnect recovery.
Why do you need buffer? Because a single network read may not correspond to one complete event — it could be half an event, or contain multiple; and UTF-8 characters, newlines, and event boundaries can all span read chunks.
Here's a parsing skeleton for demonstration:
// Runnable skeleton: POST chat and parse SSE
async function streamChat(
body: unknown,
onEvent: (event: StreamEvent) => void
) {
const response = await fetch('/api/chat', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body)
})
if (!response.ok || !response.body) {
throw new Error(`Request failed: ${response.status}`)
}
const reader = response.body.getReader()
const decoder = new TextDecoder() // SSE mandates UTF-8
let buffer = ''
while (true) {
const { done, value } = await reader.read()
if (done) break
buffer += decoder.decode(value, { stream: true }).replace(/\r\n/g, '\n').replace(/\r/g, '\n')
let boundary: number
while ((boundary = buffer.indexOf('\n\n')) >= 0) {
const block = buffer.slice(0, boundary)
buffer = buffer.slice(boundary + 2)
// ": ..." is a comment line; no data:, if empty string, skip
const data = block
.split('\n')
.filter(line => line.startsWith('data:'))
.map(line => {
const value = line.slice(5)
return value.startsWith(' ') ? value.slice(1) : value
})
.join('\n')
if (data) onEvent(JSON.parse(data) as StreamEvent)
}
}
}
// How to handle events after receiving (schematic, not complete UI)
function handleStreamEvent(event: StreamEvent) {
switch (event.type) {
case 'text_delta':
appendAssistantText(event.delta) // append text, don't replace whole
break
case 'tool_started':
showStep(event.callId, event.tool, 'running')
break
case 'tool_finished':
finishStep(event.callId, event.status, event.summary)
break
case 'interrupt':
// normal waiting for human: pop confirmation; save interruptId for resume
openConfirmCard(event.interruptId, event.payload)
break
case 'done':
markRunDone(event.messageId)
break
case 'error':
showError(event.code, event.message)
break
default:
break
}
}
// User clicks confirm: send resume separately, not a new "chat" message
async function approveInterrupt(runId: string, interruptId: string) {
await fetch(`/api/runs/${runId}/resume`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
interruptId,
decision: 'approve',
idempotencyKey: `resume_${interruptId}_1` // fixed key for same confirm action, prevent double click
})
})
// if resume still uses SSE for progress: send another request to reattach SSE, or reuse same connection per your convention
}
SSE disconnect: auto reconnect ≠ progress recovery
While streaming, if there's no business event for a long time, nginx or a gateway might disconnect due to idle timeout. You can periodically send a line starting with : as a heartbeat:
: keep-alive
The heartbeat interval should be determined by your actual gateway, load balancer, and proxy configuration; a common approach is to test starting from 15–30 seconds, then adjust according to the environment. Don't treat one interval as a fixed value for all deployments.
But a heartbeat only solves part of the idle connection problem, it doesn't guarantee the connection never breaks, nor does it replace task state persistence.
This article uses POST + fetch to read the stream, and three things need to be clear:
- If the connection breaks, the browser won't auto reconnect like native
EventSource. - Even if
EventSourceauto reconnects, it doesn't mean the server has resent the missed events. - If the server hasn't saved historical events, a
sequencealone can't recover lost progress from thin air.
You can choose different approaches based on project stage:
| Approach | Implementation cost | Capability after disconnect | Suitable scenario |
|---|---|---|---|
| Save only Run and final result | Low | Can only re-query current state or final result | Simple Q&A, first version |
| Save Run, checkpoint, and interrupt records | Medium | Can restore task state and pending approval actions, but not necessarily replay every progress event | Long-running tasks, human approval |
| Persist events and support replay by sequence | High | Can resend events the client missed | Tasks with high progress continuity requirements |
| Persist events + multi-instance push coordination | Higher | Supports event recovery and distribution across multi-instance deployments | High-concurrency production systems |
For the first version, you usually don't need to do full event replay right away. You can first ensure Run state and final results are reliable, then decide whether to persist all events based on business requirements for progress continuity.
If you need to resend progress, the server should read persisted events by the client's last confirmed sequence and replay them in order. You also need to consider event deduplication, retention period, sequence scope, and how to avoid missed events and order issues between replay and real-time push.
If using native EventSource, the SSE id: field can work with Last-Event-ID to tell the server the ID of the last event the client received. But this is only part of the recovery protocol: the server is still the one that actually resends the events.
HITL: pause for human, don't treat it as an error
When parameters are missing, approval is needed, or a high-risk operation is executed, this Run should enter interrupted, not be directly marked as failed.
The frontend shows "waiting for confirmation" or "please supplement parameters", not "execution failed". The semantics are completely different: failure means the current execution did not complete normally; interruption means the task cannot continue for now and needs external input to resume.
When pausing, at least two types of information should be left:
- Interrupt record: tells the frontend what confirmation card to show, and lets the resume API locate this interruption.
- Checkpoint: saves the minimal state needed to continue execution, e.g. the pending step and necessary parameter references.
Interrupt records and checkpoints
type Interrupt = { // interrupt record (waiting for human confirmation / supplement params)
id: string // interrupt id, must be returned as-is on resume
runId: string // owning Run
type: 'confirm_action' | 'select_option' | 'provide_input' // confirm / select / supplement
status: 'pending' | 'resolved' | 'expired' | 'cancelled' // only pending can resume
payload: unknown // content to show on confirmation card (masked)
expiresAt: string // after this time, "agree" can no longer be clicked
}
type Checkpoint = { // checkpoint: minimal state needed to continue
runId: string
nextStep: string // which step to continue from after resume
safeToolArgs: Record<string, unknown> // masked args, re-authorized before resume
conversationVersion: number // prevent conversation being changed by others during resume
createdAt: string
}
payload should not contain database connection strings, access tokens, full tool parameters, or internal stack traces. The checkpoint should only keep the fields truly needed to continue.
Resume API: what exactly should happen after user confirmation
When the user clicks "confirm, reject, or supplement params", call the resume API, not a new chat message.
// POST /api/runs/:runId/resume request body
{
interruptId: 'int_456', // still pending interrupt id
decision: 'approve', // approve | reject | supplement params etc.
idempotencyKey: 'resume_int_456_1' // fixed key for same confirmation action, prevent double-click
}
The server cannot directly trust the old parameters in the interrupt. In the skeleton below, replace authorize / reloadResource / continueFrom with your own implementations, but don't change the order:
type ResumeInput = {
runId: string
interruptId: string
decision: 'approve' | 'reject' | { fill: Record<string, unknown> }
idempotencyKey: string
}
// 1) find still-pending interrupt 2) authorize with current login state 3) prevent duplicate submission
// 4) if reject, only change status 5) reload resource 6) re-authorize 7) continue from checkpoint
async function resumeRun(input: ResumeInput, auth: AuthContext) {
const interrupt = await interruptRepo.findPending(input.interruptId)
if (!interrupt || interrupt.runId !== input.runId) {
throw new Error('INTERRUPT_NOT_FOUND')
}
await authorize(auth, interrupt) // identity only trusts this request's login state
await assertNotExpired(interrupt) // check expiresAt; reject if expired
await idempotency.acquire(input.idempotencyKey) // same key second time: return previous result, don't execute again
if (input.decision === 'reject') {
await markRejected(interrupt) // only change status
return { status: 'cancelled' as const } // don't call tools that change data
}
const checkpoint = await checkpointRepo.load(input.runId)
const latestResource = await reloadResource(checkpoint) // instance may have changed while waiting
await revalidate(latestResource, auth)
return continueFrom(checkpoint, input.decision) // continue from nextStep
}
From when the confirmation pops up to when the user clicks confirm, both resources and permissions can change. So you must reload resources and re-authorize. When the user clicks reject, only update the interrupt and Run status; don't execute write operations.
Preventing double-click is just the first step: how to avoid repeated dangerous operations
The user double-clicks "confirm", network jitter causes retries, two tabs each click once — all these can cause the resume API to receive multiple requests.
There are two common defensive layers:
Idempotency key
idempotencyKey: the same confirmation action should use the sameidempotencyKey. The server records the processing state and result for this key; duplicate requests should return the existing result, not start a new operation. Different confirmation actions must not reuse the same key arbitrarily, otherwise two legitimate operations might be mistaken as the same request.Conditional update when changing interrupt status: before actually executing the write operation, use a conditional update in code — "only if this interrupt is still
pending, change it toresolved". If two requests arrive almost simultaneously, only one update in the database will succeed (affected row count = 1); the other gets affected row count = 0, meaning someone else already handled it, so this request must not call the restart tool again.
// Written in the resume endpoint, executed via a database client (schematic)
const result = await db.query(
`UPDATE interrupts
SET status = 'resolved', resolved_at = NOW()
WHERE id = $1 AND status = 'pending'`,
[interruptId]
)
if (result.rowCount === 0) {
// already handled, expired, or cancelled → return directly, forbid running the tool
return alreadyHandledResponse()
}
// rowCount === 1: this is the first request to grab it, only then continue reloading resources and executing the restart
Just remember one thing: when two requests come together, use the conditional update above to let the database pick the first successful one; only that one can continue the dangerous operation, the other returns directly.
SSE disconnected, should the backend stop?
This is another easy place to design wrong.
The streaming connection breaking only means the frontend temporarily loses progress; it does not mean the user cancelled the task. Look at the task nature:
- Query-type: can be cancelled when the client disconnects, saving model and tool costs.
- Publishing, approval, instance restart, etc.: usually should be decoupled from the HTTP connection — the backend Run continues, results written to the database; after refresh, the user can read the state back.
// Default is to only stop pushing; whether to cancel the Run depends on policy
// Node: don't listen to req's close (may trigger after POST body read); listen to res's close
// Normal res.end() also triggers close: use !writableFinished to distinguish "client disconnected early"
res.on('close', () => {
subscriber.detach(runId) // stop pushing; not equal to user cancel
if (!res.writableFinished && runPolicy.cancelWhenNoSubscriber) {
// only client disconnected early; write / approval operations usually don't cancel on disconnect
runController.cancel(runId)
}
})
If the task is decoupled from the connection, after the user refreshes the page, they can re-query the Run status; if the task is complete, read the result directly; if still running, re-subscribe to progress according to convention.
How does it compare to LangGraph: what a framework can and cannot solve
Many teams use LangGraph to orchestrate Agent loops. It solves the same class of problems as this article, but at a different layer.
- This article discusses the external runtime protocol: conversation, Run, SSE business events, resume API, and how the frontend understands task state.
- An orchestration framework like LangGraph handles graph execution and state management: for example, how nodes run, when to interrupt, and how to save/restore graph state via Checkpointer.
You can understand the relationship as:
| Design in this article | Common corresponding concept in LangGraph |
|---|---|
| Run and checkpoint | Graph execution state and Checkpointer |
| Stable task identifier | Thread identifier, e.g. thread_id |
| HITL pause | interrupt and corresponding resume mechanism |
| SSE business events | Application-layer conversion and wrapping of execution stream |
| Authorization and resource re-check on resume | Responsibility of application service and Tool Runtime |
| Idempotency control of external side effects | Responsibility of tool execution layer and external system integration |
Specific APIs, parameters, and resume semantics depend on the actual LangGraph version you use.
Important to note: the framework saving graph state does not mean external tool calls are naturally idempotent; the framework pausing execution does not mean the frontend confirmation API, permission checks, and event replay are already implemented.
If you're already using LangGraph, you can prioritize reusing its state management and interrupt recovery capabilities; if you're building your own Runtime, you can also design Run, Checkpoint, and recovery mechanisms yourself. The key isn't whether you use a framework, but whether the state boundaries across the whole pipeline are clear.
How to test, common pitfalls, and rollout order
Before going live, at least cover the following tests:
- SSE: one event split into multiple chunks, multiple events in one chunk, Chinese split across chunks, gateway idle timeout, network disconnect, page refresh, reconnect and see if the same step is drawn again.
- HITL: double-click "confirm", confirmation expiry, permission revoked before resume, instance changed by someone else while waiting, cancel then resume.
- Conversation context (also): whether "it / the one just now" is identified correctly; whether a failed tool call mistakenly writes to
state.
Rollout order (suggested, not industry standard):
- First do POST + SSE, unified business events, frontend buffer parsing; write messages and tool results to database.
- Then add Run, interrupt, checkpoint, separate resume API, double-click prevention, re-authorization before resume.
- Event sequence numbers, pulling missed progress by number after disconnect, multi-instance progress pushing, summaries and conversation
statecan be done later.
After refresh or disconnect, you can still query where the task stopped, whether the confirmation card can still be clicked, and what should be re-validated before resuming — that's closer to being production-ready.
Summary: it's not hard to get an Agent running, the hard part is running safely after interruption
What this article most wants to convey is not some SSE API or orchestration framework, but the boundaries a production Agent must handle well.
Common misconceptions:
- Treating "waiting for human confirmation" as an error: interruption is a normal business state, while failure is a different state.
- Keeping waiting state only in memory: after process restart or scaling, the task may not be recoverable. Run, interrupt records, and checkpoints need reliable persistence.
- Executing with old parameters directly after confirmation: resources and permissions can change during the wait, so they must be re-read and checked before resume.
- Treating SSE as the task itself: SSE pushes progress, but task state is determined by persisted Run and execution records.
- Believing one SQL can guarantee an external operation is executed only once: the database can control concurrent claiming, but external side effects also need idempotency, state verification, and failure recovery.
- Stuffing the entire chat history into the model: costs more and is more susceptible to interference from old information. Distinguish reliable facts, historical summaries, and recent verbatim turns.
Finally, remember three things:
- Push progress with SSE business events, don't make the frontend guess the task's step by reading natural language.
- Waiting for human confirmation is a normal flow, not an error; after user confirmation, go through the resume API, carrying
interruptIdto continue this Run, don't treat it as a new chat round. - Connection broken ≠ task gone: Run / interrupt / checkpoint should be written to database; before resume, re-verify permissions, re-read resources, and prevent double execution.
This article expands on common problems in Agent deployment, hoping to give some help and reference to those building related systems. Any questions or different views are welcome in the comments.
Top 1 of 3 from juejin.cn, machine-translated. The original thread is authoritative.
Drop the repo link so we can check it out.
That's right, not publishing the code leaves us with only theory and no way to see the entire runtime process.
Hang on, still putting together a demo.