The AI Chat Box Is a Trap: Four Layers That Separate a Demo from a Real Product

OK, OK, hello everyone, welcome to Dapeng AI Education. I'm Zhang Dapeng.

Recently, many frontend developers have been exploring AI application development. The first projects they create are often quite similar: a conversation list on the left, a chat window in the center, an input box at the bottom, connected to a large model API.

This demo can be completed in a day or two, but there's still a long road ahead before it becomes a truly shippable AI product.

While reviewing a React 19 + TypeScript AI assistant project recently, I became increasingly convinced of one thing:

The watershed moment for a frontend developer transitioning to full-stack AI isn't whether they can draw a chat box, but whether they can handle streaming events, disconnection recovery, tool approval, and consistency.

A typical web application is triggered by a user click, sends a short request, the server returns a result, and the page updates. An agent application, however, might run continuously for tens of seconds or even minutes, calling search functions, databases, or external devices along the way.

A single response is no longer a string but a constantly changing event stream that may pause and resume.

This article, starting from real code, breaks down the layers of capability that a shippable AI frontend must at least complete.

1. Why the "Typewriter Effect" Is Not a Streaming Architecture

The streaming effect in many AI chat demos simply chops the text returned by the server into small pieces and appends them to the page character by character:

const reader = response.body?.getReader();
const decoder = new TextDecoder('utf-8');

while (reader) {
  const { done, value } = await reader.read();
  if (done) break;
  setAnswer((text) => text + decoder.decode(value));
}

This code can achieve a visual "generate and display simultaneously" effect, but it cannot yet support a real Agent.

Because a real stream contains more than just text; it may also include:

• 💬 answer.delta: A new segment of the answer.
• 🔧 tool.call: The model is preparing to call a tool.
• ✋ tool.approval.required: An important operation is waiting for user approval.
• ✅ tool.result: The tool execution is complete.
• 📚 research.progress: The stage progress of a research task.
• 🛑 error: The current task has failed.
• 🏁 end: The entire round of messages has ended.

If the frontend treats all data as strings, it will subsequently be forced to keep writing conditional judgments, eventually turning into a set of chat box patches that are difficult to maintain.

The correct approach is to first convert the network byte stream into typed events, and then let different state modules consume those events.

2. Layer 1: Reading SSE with fetch + ReadableStream

The project does not directly use the simplest EventSource but uses fetch to read SSE.

The reason is practical: the business requires carrying authentication information, actively canceling requests, and uniformly handling connection status.

The connection request looks roughly like this:

const response = await fetch(url, {
  headers: {
    Accept: 'text/event-stream',
    Authorization: `Bearer ${token}`,
  },
  signal: controller.signal,
});

if (!response.ok || !response.body) {
  throw new Error(`SSE connection failed: ${response.status}`);
}

Then, ReadableStream and a UTF-8 decoder are used for continuous reading:

const reader = response.body.getReader();
const decoder = new TextDecoder('utf-8');
let buffer = '';

while (!signal.aborted) {
  const { done, value } = await reader.read();
  if (done) break;

  buffer += decoder.decode(value, { stream: true });
  // Split complete SSE records by blank lines; half-received messages stay in the buffer
}

There is a detail here that is easily overlooked: one reader.read() does not guarantee exactly one complete event.

The network layer might split one event into two segments, or merge multiple events into the same data block. Therefore, the frontend must keep a buffer and only consume records that have been fully received.

SSE Records Have More Than Just Data

A recoverable SSE typically contains:

id: 42
event: tool.approval.required
data: {"conversation_id":"...","call_id":"call-1"}

After parsing, the frontend should get a unified object:

type SSEEvent = {
  id?: string;
  type: string;
  data: unknown;
};

Next, it is handed over to Redux or another state container via a central dispatcher:

switch (event.type) {
  case 'tool.approval.required':
    dispatch(addPendingApproval(event));
    break;
  case 'tool.approval.cleared':
    dispatch(resolveToolApproval(event));
    break;
  default:
    dispatch(sseEventReceived(event));
}

This way, chat content, approval notifications, research progress, and system status don't all have to be crammed into the same component.

3. Layer 2: Where to Resume After Disconnection

Being able to automatically reconnect is not the same as being able to recover.

If the connection drops midway and the frontend re-requests the same address without telling the server where it has already received data, two things can happen:

• 🔁 Replay from the beginning, causing duplicate content and duplicate notifications on the page.
• 🕳️ Only receive new content, permanently losing events that occurred during the disconnection.

Solving this problem requires an event cursor.

Every time the frontend receives an event with an id, it persists the latest cursor:

onEvent(record);

if (record.id) {
  dispatch(sseLastEventIdAdvanced(record.id));
}

On the next connection, the cursor is passed to the server:

const params = new URLSearchParams();

if (lastEventId) {
  params.set('last_event_id', lastEventId);
}

fetch(`/api/messages/${messageId}/events?${params}`);

The project puts the cursor in the query parameters rather than a custom request header to keep cross-origin GET requests simple and avoid extra preflight issues.

The server simultaneously supports the standard Last-Event-ID request header and the query parameter, then reads data after the cursor from the event log.

At this point, SSE upgrades from a "typewriter animation" to a truly recoverable transport layer.

What If the Events Have Already Been Cleaned Up

Event logs cannot be stored indefinitely. If the client is offline for too long, its saved cursor may have fallen outside the retention window.

The server will return an event like backlog.truncated. Upon receiving it, the frontend cannot continue to trust the old cursor and should:

• 🧹 Clear the local Last-Event-ID.
• 🔄 Re-fetch the current complete state.
• 🚫 Avoid continuing to reconnect from a position that no longer exists.

This kind of "cursor invalidation" branch often reflects whether an AI product has truly considered production environments more than the normal connection path does.

4. Layer 3: Tool Calls Cannot All Be Executed by Default

The biggest difference between an Agent and a regular chatbot is that it doesn't just generate text; it may also execute actions.

Querying weather or reading public documents carries low risk, but modifying a database, sending messages, deleting files, or operating external devices cannot be executed immediately just because the model generated parameters.

The basic flow for human-in-the-loop approval is:

Model requests a tool
  ↓
Backend determines the action requires approval
  ↓
Saves the pending approval state
  ↓
Sends tool.approval.required
  ↓
Frontend displays parameters and risks
  ↓
User approves or denies
  ↓
Resumes the Agent along the original session

What the frontend submits is not a natural language "I agree," but a structured decision:

type ToolAction = {
  call_id: string;
  decision: 'approved' | 'denied';
  comment?: string;
};

Approval and denial must be bound to a specific call_id; otherwise, when multiple tools appear in a single round of conversation, the system cannot determine which action the user actually approved.

5. Layer 4: Pause State Must Be Written to the Database

Saving pending approval tools only in memory is a very dangerous implementation.

The user might take several minutes to decide, during which the service could restart, scale out, or route the next request to a different instance. If the approval state only exists in some Python object, the original task is already lost when the user clicks "approve."

The project uses pending_tool_state to save:

• 🆔 Session ID and user ID.
• 🧠 The message context needed to continue reasoning.
• 🔧 The pending tool call and its parameters.
• 🧰 The currently available tools and their schemas.
• 🤖 Agent configuration.
• ⏰ Creation time, expiration time, and recovery status.

When saving, "session + user" is used as the unique boundary:

INSERT INTO pending_tool_state (...)
VALUES (...)
ON CONFLICT (conversation_id, user_id)
DO UPDATE SET
  pending_tool_calls = EXCLUDED.pending_tool_calls,
  status = 'pending';

After the user submits a decision, the system first atomically changes the status from pending to resuming:

UPDATE pending_tool_state
SET status = 'resuming',
    resumed_at = NOW()
WHERE conversation_id = :conversation_id
  AND user_id = :user_id
  AND status = 'pending';

Only one request can successfully claim the right to resume, thus preventing the user from double-clicking the button or network retries causing the tool to execute repeatedly.

After the task successfully ends, the pending approval record is deleted; if the process is abnormal during recovery, a background cleanup task can reorganize the stuck state based on time.

6. The Approval UI Is Not as Simple as Two Buttons

A trustworthy approval card should at least show the user:

• 🏷️ The tool name and action type.
• 📦 The key parameters about to be submitted.
• ⚠️ Whether it will write data, call external systems, or incur costs.
• 📝 An optional approval/denial note.
• ⏳ Current status: waiting, resuming, approved, denied, or expired.

The "approve" button should also not disappear from the page immediately after being clicked.

A more reasonable state change is:

awaiting_approval
  ↓
submitting_decision
  ↓
resuming
  ↓
completed / failed

If the server later sends tool.approval.cleared, the frontend must also clean up old notifications to prevent the user from operating on an already invalidated approval from a historical notification.

This is also why the approval state should enter the global event system, rather than being stored only in the local useState of the current chat bubble.

7. The Idempotency Key Is the Last Line of Defense

Even if the database can atomically claim the pending approval state, the frontend should still carry an idempotency key when submitting write operations:

await fetch('/stream', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Idempotency-Key': requestId,
  },
  body: JSON.stringify({
    conversation_id: conversationId,
    tool_actions: actions,
  }),
});

It mainly solves the gray state of "not knowing whether the last request has already reached the server."

For example, the user clicks approve and the network immediately disconnects. The frontend receives no response, but the server may have already started executing. If the frontend regenerates a request ID and submits again, it could cause a duplicate operation.

The correct approach is: reuse the same idempotency key when retrying the same logical action.

8. What Should Tests Verify

This project has already organized streaming UX and approval recovery into independent test boundaries. The focus is not "whether the page displays a few words," but the following contracts:

• 🧪 Every non-empty line of SSE can only be data:, id:, or event:.
• 🏁 The stream can ultimately receive a clear end event.
• 💾 Session data is truly written to the database.
• ✋ Pending approval records can transition from pending to resuming.
• 🧹 After successful recovery, pending_tool_state is cleaned up.
• 📴 If the user cancels the stream midway, no orphaned approval state is left behind.
• 🔄 The same session can subsequently continue to initiate requests.

The corresponding verification entry points include:

# Backend approval, event replay, and session recovery
python -m pytest -q `
  tests/test_tool_approval.py `
  tests/test_event_replay.py `
  tests/test_continuation.py

# Frontend event stream and event dispatch
cd frontend
npm test -- `
  src/events/eventStreamClient.test.ts `
  src/events/dispatchEvent.test.ts

It must be honestly stated: in the environment where I am currently checking the code, the project's pytest and vitest dependencies were not installed, so I only completed the source code and test contract verification, and did not package these two commands as "passed locally."

This itself is part of engineering practice: verification that has not been successfully executed cannot be written as a green light.

9. The Real Capabilities Frontend Developers Need to Supplement for Full-Stack AI

Looking back from this project, frontend developers do not need to learn all backend frameworks first. A more effective route is to build capabilities around a complete user journey:

• 🌊 Able to parse and manage structured SSE, not just display streaming text.
• 🧭 Able to use event cursors to complete disconnection resumption and state reconstruction.
• 🧱 Able to map Agent state to a clear frontend state machine.
• ✋ Able to design human-in-the-loop approval for important tools.
• 💾 Understand why pause state must be persisted.
• 🔐 Understand idempotency, authorization, and duplicate execution risks.
• ✅ Able to use end-to-end evidence to prove that a task has truly been completed.

When you can independently complete these capabilities, what you are building is no longer a "chat page wrapping a model API," but an AI product with genuine operational boundaries.

Final Words

The AI era has not made the frontend less important. On the contrary, the stronger the model's capabilities, the more the frontend needs to turn an unpredictable execution process into a product experience that users can understand, control, and recover from.

The chat box is just the entry point.

The event stream determines whether the experience is continuous, the approval flow determines whether the system is trustworthy, and persistence and idempotency determine whether it can safely run in production.

Next, I plan to continue expanding this chain towards "visible costs, explainable failures, and verifiable results." For frontend students wanting to transition to AI application engineering, these capabilities will be closer to the real job requirements than memorizing another set of component APIs.