跪拜 Guibai
← Back to the summary

The Session Service Your Agent Can't Skip: Identity, Idempotency, and Real Cancel

🚀🚀 [End of Batching 6] Agent Session Service and Auth Context: Making Production Agents Truly Controllable

Preface: The closed beta has come to an end, and I'm starting a systematic retrospective: How does a production Agent actually run? I'll cover pitfalls, experience, and implementation across a few articles. This is part 6: Agents often fail not because the model is dumb, but because identity is lost, sessions are kept only in memory (lost on restart or machine change), and when a user clicks cancel, model and tool requests keep running.

I hope this article helps those who are implementing Agents in production. Feel free to share your practices in the comments.

In production, these problems often stack together:

These issues seem unrelated but all point to the same core: An Agent isn't just one model call; it needs reliable session, identity, and task lifecycle management.

This article explains four things:

  1. Who is talking: tenant (the isolation boundary for a company or customer in the system) and user identity must come from trusted login state and be written into the database's session and Run (a task created for one click of send).
  2. Whose permissions call tools: The entire call chain must pass the same trusted request context; don't use userId from model parameters to determine who the user is.
  3. One click should not create two tasks: Whether it's double-click send or automatic retry after network timeout, it should return to the same Run.
  4. What does "cancelled" really mean: Don't just close frontend streaming; interrupt in-flight model and tool requests where possible, and confirm the task ends in the correct state.

Using the OpsPilot (fictional enterprise ops assistant) case, tech stack is TypeScript. Go, Java, Node.js etc. can also implement this. The key is to separate responsibilities and boundaries: Don't let the frontend hold the model API key directly.

Snipaste_2026-10-27_16-01-34.png

What does the session service manage?

The session service must handle at least five things:

  1. Session and message persistence: Write sessions and messages to DB, support creation, append, and paginated queries.
  2. Run lifecycle management: Manage task states such as queued, running, waiting for human confirmation, completed, failed, and cancelled.
  3. Trusted identity context propagation: Convert login state into an explicit request context and pass it to orchestration and tool layers, not using userId from model parameters.
  4. Idempotent Run creation: Prevent double-clicks, network retries, etc., from executing the same request twice.
  5. Task cancellation and interruption: Not only notify frontend to stop displaying, but also interrupt in-flight model and tool requests where possible.

Why emphasize the session service?

Because its responsibilities differ by scenario:

These scenarios have different business logic but all need reliable session and task management.

Even when using frameworks like LangGraph to run the agent loop, tenant isolation, login state, cancellation, and the interface with the frontend are still problems the session service must solve. The framework can track which step the graph is at, but whether the task is running, whether it's cancelled, and whether the current user has permission to operate on this task all need clear, trusted management.

Don't let framework state and the Run state in the business database disagree; otherwise debugging is painful: the framework thinks the task is still running, but the frontend already shows cancellation successful.

Request Context: The Whole Chain Trusts Only This One

When a request enters the session service, first build a RequestContext object based on the verified login state. Subsequent orchestration and tool calls use this trusted context, passing it explicitly via parameters when necessary. Don't reconstruct identity from scattered headers at each layer, and don't trust the model's own "who I am" claim.

type RequestContext = { // request context after entry authentication
  requestId: string         // request id for logging and debugging
  traceId: string           // trace id shared across session/tool/model
  tenantId: string          // tenant; for DB queries and retrieval filtering
  userId: string            // current user; tool auth reads here
  roles: string[]           // roles for permission and visibility
  scopes: string[]          // allowed tool or resource scopes
  authHeader?: string       // server-side only, not in prompt or plaintext logs
  locale?: string           // optional: language/region
  deadlineAt?: number       // run deadline; cancel when reached
}

Three points to remember about this object:

A detail often missed: RequestContext is a trusted data structure inside the server, but it doesn't mean every field should be passed directly to the model, nor that every field can be trusted unconditionally.

For example, tenantId and userId come from authentication results; roles and scopes from trusted authorization logic; traceId can be used for tracing but not for permission decisions.

sequenceDiagram
    participant C as 前端
    participant S as 会话服务
    participant R as Agent Runtime
    participant T as 工具

    C->>S: POST /chat
    S->>R: 读历史、开始 Run
    R-->>S: 进度事件
    S-->>C: SSE 推进度
    R->>T: 带 RequestContext 调工具
    T-->>R: 工具结果
    C->>S: POST /cancel
    S->>R: 取消 Run（写库 + abort）

Three more things about IDs, cancellation, and history:

Snipaste_2026-10-27_16-01-42.png

Auth, Multi-tenancy, and Credentials

Identity checks should be done in at least three layers, not just at the request entry. Reason: entry only verifies "who sent the request"; at execution time you also need to verify "does this user have permission to execute this action".

Layer Responsibility
Gateway/Session service Validate login state, build RequestContext, rate-limit if needed
Agent Runtime (orchestration) Decide which tools can be called based on scopes, pass trusted RequestContext to the code that actually executes tools
Tool implementation Re-validate "can this user do this" per business rules; for write operations, record operator, time, and result

Common incident: entry validated login, but the tool directly uses userId from model parameters for auth. If the model is influenced by prompt injection, it may generate another user's ID, making the tool think it's operating on behalf of another user.

Therefore, user identity in tool parameters cannot be the auth basis; only the RequestContext built by the server's trusted auth flow can be used.

DB queries must include tenant

Tenant ID and user ID should be written into business tables, and every query must include the isolation conditions. Don't rely on the frontend's convention of "don't request someone else's conversation ID".

CREATE TABLE conversations (
  id UUID PRIMARY KEY,                 -- conversation id
  tenant_id TEXT NOT NULL,            -- tenant; required in queries
  user_id TEXT NOT NULL,              -- owner user; required in queries
  title TEXT NOT NULL,                -- conversation title, e.g. 'Troubleshoot - payment-api'
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- For listing conversations by tenant + user
CREATE INDEX conversations_tenant_user_idx
  ON conversations (tenant_id, user_id, created_at DESC);

Avoid providing a generic query function that only accepts conversationId. Otherwise, if someone guesses or obtains another user's conversation ID, they might attempt to read data across boundaries.

Better to match three conditions simultaneously:

type ConversationScope = {
  tenantId: string
  userId: string
}

async function getConversation(
  db: Database,
  scope: ConversationScope,
  conversationId: string
) {
  // match id + tenant + user, all required
  return db.oneOrNone(
    `SELECT * FROM conversations
     WHERE id = $1 AND tenant_id = $2 AND user_id = $3`,
    [conversationId, scope.tenantId, scope.userId]
  )
}

One business boundary to note: Not all users only access their own conversations. If the product supports team shared conversations, project collaboration, or tenant admins viewing data, add explicit sharing relations and authorization rules in queries, rather than simply removing the userId condition. Isolation should be determined by business permission, not by whether an ID exists.

The same isolation principle applies beyond chat tables; otherwise data still leaks from other access points:

Why emphasize "isolate before query"?

Because querying all data first and filtering at the application layer doesn't guarantee unauthorized data was never touched. That data may have entered sorting, cache, logs, or even model context. For enterprise knowledge bases or internal ops assistants, this can directly become cross-tenant data leakage.

Admins who need cross-tenant debugging should use separate admin interfaces and authorization flows, granting access on demand, and logging operator, reason, and scope. Don't reuse the ordinary chat API.

After user logout or tenant deactivation, there must be clear data retention, deletion, and index cleanup policies. Sessions, memories, uploaded files, and retrieval indexes all need management and verification of cleanup.

Snipaste_2026-10-27_16-01-54.png

Keys should never enter prompts

Keys and access credentials should only be managed server-side, not written into prompts sent to the model, nor into persisted chat bodies or message history.

Credential type Storage and usage
API key for the model In a key management system or protected server config, authorized only to services that call the model
User access credentials Held securely server-side, swapped or used to call business APIs as needed; never stuffed into chat history
Tenant's own plugin keys Stored per-tenant, encrypted; decrypted server-side only when actually calling the plugin
Identity credentials for internal business systems Server obtains appropriate credentials based on RequestContext and authorization policy, and performs necessary permission checks at tool side

Emphasize again: Fields like userId, tenantId, roles in model tool parameters cannot be used as trusted identity and authorization basis. Tools should ignore those fields for auth and only trust the server-built RequestContext and formal authorization results.

One more detail: Even if the model is not malicious, user input, prompt injection, or context confusion can generate untrusted tool parameters. Therefore, the security boundary cannot be built on the assumption that "the model will fill in correctly."

Starting a Run: One Click Creates One Task

When a user clicks send, the request body must carry at least three things: conversation ID, user's original message, and an idempotency key (also called request deduplication key).

This key is generated by the client when the user initiates the send. Browser or app can generate it, but on network retry for the same send, the original key must continue to be used. Don't generate a new key on each retry; otherwise the server can't tell these requests belong to the same operation.

For example, user clicks send, the network request reaches the server, but the response is lost on the return. The frontend thinks it failed and sends again. If the server has no idempotency, it may start a second agent loop, causing duplicate model calls and even repeated real business operations from tools.

type StartChatInput = {
  conversationId: string
  idempotencyKey: string // request key; same key in short time returns same Run
  message: string        // user's current input
}

The key point is not to check whether "two messages are identical" but whether "two requests are the same business operation". Users might intentionally send two identical messages, so don't deduplicate based on message text.

Within a short time, the same request key under the same tenant must map to the same Run. The server should also verify the key's association with the original request: if the same key is submitted with a different conversation or message, reject the request or return a clear conflict error, rather than silently binding new content to an old task.

Agents can run long, so don't make the HTTP request that receives the send wait for model and tools to finish. Split into two phases:

  1. Register the task first: Create a Run in the database and use a unique constraint to prevent duplicate creation for the same request key. Insert success means a new task; if the key already exists, return the existing runId instead of starting the agent again.
  2. Hand to background execution: Pass the runId to a background queue or task scheduler, and the Worker calls the model and tools. The receiving side returns task info and pushes progress via SSE.
CREATE TABLE runs (
  id UUID PRIMARY KEY,                          -- Run id
  tenant_id TEXT NOT NULL,                     -- tenant; unique constraint with request key
  conversation_id UUID NOT NULL,               -- which conversation
  idempotency_key TEXT NOT NULL,               -- client request key; same key same Run in short time
  status TEXT NOT NULL,                        -- queued / running / interrupted / ...
  cancel_requested BOOLEAN NOT NULL DEFAULT FALSE, -- whether user requested cancel; background checks each step
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  UNIQUE (tenant_id, idempotency_key)           -- same tenant, same key only one Run
);

// Step 1: Register Run; if same tenant+key resubmits, return existing runId
// Step 2: Notify background execution; this function doesn't run the full Agent
async function startRun(input: StartChatInput, ctx: RequestContext) {
  const run = await db.insertRun({
    tenantId: ctx.tenantId,
    conversationId: input.conversationId,
    idempotencyKey: input.idempotencyKey,
    status: 'queued' // registered, waiting for background
  })
  await queue.publish('agent.runs', { runId: run.id, tenantId: ctx.tenantId })
  return run
}

This shows the basic flow, but in production there's a failure window often missed:

A successful DB write does not guarantee a successful queue publish.

Suppose the Run is written to DB, but the service crashes just before queue.publish(). That Run may stay in queued forever. Conversely, if the message is redelivered, multiple Workers might simultaneously try to execute the same Run.

Production systems typically need additional mechanisms:

That's why you can't just do an insert plus a publish and assume task scheduling is reliable.

Timeout layering (recommended):

Especially distinguish: Request idempotency is not tool execution idempotency. Even if the same request creates only one Run, re-executing a tool after a Worker crash can still repeat operations. So for side-effectful tools, design independent idempotency keys, execution records, or state checks based on business logic.

Cancellation Must Actually Stop Model and Tool Requests

An SSE or HTTP connection closing only means the frontend temporarily stops receiving progress; it doesn't mean the user explicitly requested cancellation, and it doesn't mean the backend task has stopped.

For example, a user clicks "cancel" in an ops agent, the loading animation disappears, but the model request and tool HTTP requests running in the background may continue. For long-running tasks, this both incurs extra cost and can mislead the user into thinking the operation stopped while the business is still changing.

So when the user clicks cancel, the frontend should call an explicit cancel endpoint, not just close the streaming.

The server must do at least two things:

  1. Write the cancel intent to the database, e.g., set cancel_requested = true, so other Workers or subsequent execution steps can see it.
  2. Try to find the task executing this Run and trigger an interrupt signal (e.g., abort()) to interrupt interruptible model and tool HTTP requests.
async function cancelRun(runId: string, ctx: RequestContext) {
  // 1) Persist cancel intent so other machines and steps can see it
  await db.markCancelRequested(runId, ctx.tenantId)

  // 2) Attempt to interrupt local task; cross-machine execution needs additional notification
  const worker = workers.get(runId)
  worker?.abort()
}

Important: Writing cancel_requested to the database only records the intent; it does not mean the task has stopped.

If the task runs on another machine, local workers.get(runId) won't find it; if a tool has already sent a request to an external system, local interruption may not undo the external operation.

Therefore, every background step should check both the database cancel_requested and the current interrupt signal (e.g., AbortSignal). For cross-machine tasks, propagate cancellation via queue notifications, event mechanisms, or Worker polling.

The frontend cancel path can be unified as POST /runs/:id/cancel. The server should first verify the current user has permission to cancel this Run, then record the intent and notify the executor.

For state management, distinguish:

Don't directly tell the frontend the task is cancelled after receiving the cancel request. Better to return "cancel request accepted", then update the Run based on the Worker's actual state.

If a tool has already produced irreversible side effects (restart service, send email, submit order), cancellation can't turn back what happened. In that case, clearly inform the user how far the task got, and design compensation operations or manual processing flows per business needs.

Tasks can run in a background queue, but there must still be clear Run state as the basis for how far the task has progressed, how progress is pushed, and who can view and cancel it. Don't create a separate task table missing tenant fields; otherwise task status, permissions, and cancellation can easily disagree.

Snipaste_2026-10-27_16-02-01.png

External Interfaces and Troubleshooting

At minimum, prepare the following interfaces (paths can follow project conventions, but responsibilities must be clear):

Run statuses should be unified:

The frontend updates the page based on these statuses and SSE business events; don't guess progress by parsing incomplete internal logs.

Also, SSE is for real-time push, but should not be the only source of task state. Browser refresh, network switching, or mobile app entering background can break the connection. When the client reconnects, it should be able to query current status by runId and restore necessary progress from the server.

When things go wrong, check here first (the most common pitfalls):

  1. Can't continue a conversation after switching phone or browser: Check whether sessions, messages, and Runs are persisted to DB; if only in memory, they may be lost on restart or scaling.
  2. Tool returns 401, or tool wrongly uses another user's identity: Check whether RequestContext was passed to the tool layer, whether access credentials are expired, and whether code misuses userId or tenantId from model parameters.
  3. User clicked cancel but costs keep accruing: Check whether only SSE was closed; whether cancel intent was written to DB; whether model and tool requests actually support and receive abort signals.
  4. User can see conversations from other tenants or users: Check whether queries only match by conversationId; whether tenant, user, and sharing permission conditions are complete.
  5. Two tasks start after double-click send: Check whether Run creation uses the client-generated idempotency key; whether the same request key maps to only one Run; whether Workers have a concurrency claim problem.
  6. Run stays in queued state: Check whether after DB write, the queue publish succeeded; whether Workers are properly claiming tasks; whether tasks were registered but never delivered.
  7. Tool operations execute repeatedly: Besides checking idempotent Run creation, check whether the tool itself has idempotency protection, especially for orders, notifications, config changes, and ops operations.

Summary

For a session service to truly stand, the core is three things: Where identity comes from, how one send starts and stops, and how data is isolated.

  1. Trust only the RequestContext for identity. After gateway or session service authenticates, pass context through the entire call chain. Tools cannot use userId or tenantId from model parameters as auth basis, and must validate operation permissions per business rules. Keys and access credentials only live server-side, never in prompts or chat history.
  2. One click maps to one Run, and cancellation must have a clear result. The client marks the same send with an idempotency key; the server prevents duplicate task creation, persists the task, and hands it to background execution. On cancel, record the intent, notify the executor, and interrupt interruptible requests. Closing SSE only stops progress reception; it doesn't stop the task.
  3. Tenant isolation must happen during data access, not after query. Sessions, knowledge retrieval, long-term memory, and background tasks all apply isolation rules. Don't query all data first and filter at the application layer; also don't ignore easy-to-miss engineering details like Worker retries, cross-machine cancellation, and tool side effects.

Finally, two practical questions for discussion:

Feel free to share your practices, pitfalls, or questions in the comments. Especially welcome real failure cases.

I hope this article helps those implementing Agents in production.