Lomo Agent: A Persistent Execution Architecture for Creative AI That Keeps Working While You Don't
Lomo Agent: Architecture Design and Engineering Practice for Creative, Continuously Collaborative Agents
Making a product short film involves requirements communication, reference material collection, storyboarding, image generation, video generation, and repeated revisions. Some steps take seconds, some take minutes; some are better left to model decisions, some must be confirmed by the user. After the first version is done, the work is far from over.
Since the AI era began, I have rarely written technical articles, spending more time on AI exploration across different application scenarios.
Many believe the biggest beneficiaries of the AI era are product managers (PMs), because the distance between ideas and implementation has been greatly shortened. But in my view, AI empowers technical people even more deeply. It is gradually breaking the constraints that technology implementation places on creativity, giving technical people the chance to step out of code and engineering details to re-examine the essence of product, architecture, and human-computer interaction.
In the past, we spent a lot of time thinking about how to make machines execute instructions correctly; now we can finally put more energy into a more interesting question: how to make machines truly understand human intent and collaborate with people in a more natural way.
Recently, we launched Lomo Agent on Lomote AI. I'd like to take this opportunity to talk about the thinking, architecture design, and engineering practice from concept to launch.
The article will cover task organization, message routing, persistent execution, asynchronous tasks, context engineering, long-term memory, autonomous progression, and shared creative workspace. Some of these designs have been implemented, others are still being explored and validated. I will also reference DeepSeek, Manus, Claude Code, and other excellent products and technical practices, sharing my thinking and trade-offs between different options.
Lomote AI and Lomo Agent
Lomote AI is a one-stop AI creation platform we built. Interestingly, in this project I played more of a PM role, while also participating in the design and construction of the underlying infrastructure. This gave me the chance to rethink from both product and technical perspectives what an AI creation tool should look and feel like.
Currently, Lomote AI integrates multiple image, video, and 3D generation and enhancement models, and provides capabilities like asset libraries, inspiration boards, and Flows visual workflows. We want creators to manage assets, combine models, and orchestrate creative flows in a unified workspace, moving AI creation from scattered model calls to a complete, coherent creative experience.
These capabilities largely solve the fragmentation problem of AI creation tools, but as usage deepens, we gradually discovered that tool integration is only the first step. How to make them truly work together to liberate user productivity is the more worth-exploring problem.
In the traditional AI creation process, users still need to plan steps themselves, select models, organize material, and repeatedly pass context and intermediate artifacts between tasks. When the creative flow becomes complex, users often need to spend a lot of energy coordinating each link.
Lomo Agent was born in this context. I wanted AI to evolve from a mere creation tool into an intelligent agent that can understand intent, plan tasks, and collaborate continuously. Users only need to describe a goal, and the Agent can call the platform's existing models and tools, organize the creative flow, and continuously adjust based on user feedback during execution.
To achieve this experience, traditional single-turn tool calls are clearly insufficient. How to organize tasks, manage context, coordinate asynchronous execution, and how the Agent continues to advance work after the user leaves have all become architectural problems we need to solve.
Starting from a Repeatedly Modified Creation Task
Suppose a user says on Monday: "Help me make a set of vintage-style brand posters." The Agent organizes requirements and starts generating reference images. Meanwhile the user inserts another task: "By the way, turn yesterday's cat image into a five-second video." On Wednesday, the user mentions again: "Make that set of posters warmer, and adjust the text position."
This is perfectly normal in human collaboration, but for an Agent it involves parallel management of multiple tasks, asynchronous generation, context association, and historical asset retrieval. As tasks progress, there are also cancellation, retry, budget control, user preference changes, and multi-device collaboration issues.
About a month ago, I started thinking about a question: Why must an Agent be centered on sessions?
Most existing Agent products still follow the Chatbot interaction paradigm, where users need to actively create, find, and maintain sessions. For heavy users, sessions accumulate, and the context management that AI should handle becomes a burden on the user.
In real life, people don't deliberately maintain "sessions." We can switch topics anytime, or naturally continue a previous conversation days later. An ideal agent should perhaps have this ability.
So we tried to weaken the user-facing session concept in Lomo Agent, leaving task recognition, context isolation, and historical association to the system. Of course, this is only one part of the architecture. To truly support continuous creation, we still need to solve engineering problems like execution recovery, asynchronous scheduling, long-term memory, budget authorization, and shared artifact management.
While building the demo, I happened to read the DeepSeek team's architecture vision for Super-agent. It's interesting that everyone encounters similar problems in task organization and context management. The Router/Worker responsibility division in DeepSeek's article, and the subsequent message routing design, inspired me and helped refine Lomo Agent's overall architecture.
Below, I'll discuss my thinking and trade-offs on some issues, combined with actual design and implementation.
1. Task Organization: From User Requests to Sustainably Advanced Work Items
Returning to the earlier example, a user may switch topics within minutes or pick up a task days later. For an Agent, receiving a message is just the beginning; more important is understanding which goal the message belongs to and how to find the previous execution progress and related material.
This involves an architectural trade-off: if all tasks share one context, information from different goals interferes with each other and the model's context window expands quickly; if fully isolated by session, it breaks the historical association between tasks.
So in Lomo Agent, we separated interaction entry, execution context, and long-term memory into three relatively independent layers:
| Concept | Problem Solved | Lomo Agent Handling |
|---|---|---|
| Interaction Entry | Where the user expresses needs | Unified user-side main line; no manual creation/switch of sessions |
| Execution Context | What information should be jointly provided for model reasoning | System creates independent matters (tasks) based on goals, each with context boundaries |
| Long-term Memory | What information is worth retaining across matters and time | Traceable profiles, facts, and episodic summaries, recalled on demand |
Users only need to express intent naturally at a unified entry; the system handles task attribution and creates independent matters for work that needs continued advancement. Each matter has its own context, execution state, and artifact references, and when necessary can access other matters' information through controlled retrieval and sharing mechanisms.
In this design, we referenced different existing product practices.
For example, ChatGPT enhances information continuity across conversations through cross-session memory; Claude uses mechanisms like Projects to let users organize knowledge and context around specific projects. These designs have advantages: explicit sessions and project boundaries are easy to understand, manage, and share, and give users more direct control over the context scope.
But for creative scenarios requiring long-term, frequent task switching, users still need to bear some context organization work. Cross-session memory can help the Agent understand user preferences and history, but cannot fully represent an ongoing task and its progress.
Lomo Agent therefore chose to hand more task organization to the system, freeing users from routine session management as much as possible. Of course, automatic recognition may lead to task attribution errors. To this end, we kept explicit matter entry and correction mechanisms, and preferred deterministic routing when goals are clear, reducing unnecessary model judgment.
2. System Overview: Coordination Layer, Execution Layer, and Shared Workspace
After determining task organization, the next step is how to make these matters actually run.
A continuously collaborative Agent needs to handle new user requests, advance existing tasks, wait for external generation results, and respond to user modifications. If all this is handled by the same Agent, context expands quickly and tasks interfere; if each request creates an independent Agent, there's extra coordination cost.
Based on this, I designed a Main / Task dual-layer structure for Lomo Agent and introduced a persistent runtime in the execution layer, allowing different tasks to advance independently while accessing and modifying creative artifacts through a shared workspace.
2.1 Overall Architecture
The whole system can be divided into four main parts:
- Main (Mainline): The entry for continuous user interaction, responsible for understanding new requests, associating existing tasks, handling simple questions, and coordinating execution and feedback across matters.
- Task (Matter): An independent execution space that carries a specific creative goal. Each task maintains its own brief, context, execution plan, message inbox, and run state.
- Agent Runtime: Manages model inference, tool calls, async waits, state persistence, and execution recovery, checking budget, permissions, and stop conditions at each step.
- Shared Workspace: Stores creative artifacts such as images, videos, 3D, documents, asset collections, and Flows workflows, allowing users and different tasks to access and modify them within permissions.
In role division, Main mainly serves as Router, while inside Task a Worker advances the specific work. Router and Worker are logical running roles and do not require using different base models.
An important feature of this architecture is that task execution and creative artifacts have independent lifecycles.
For example, a user initiates poster creation on the main line; the system creates an independent task for it. Subsequent reference images, generation records, and modification history are maintained by that task, and the final artifact is stored in the shared workspace. Even if the user switches to other tasks, the poster task can continue waiting for generation results and resume execution when needed.
The main line only needs to maintain the task's summary and current state, not keep the full history of every model call and tool execution. This avoids unbounded expansion of the main-line context over time.
Inspiration from DeepSeek Super-agent
During design, DeepSeek's Super-agent architecture vision gave us a lot of inspiration.
It also uses the idea of separating coordination and execution, organizing different goals through Router/Worker, and maintaining task attribution through mechanisms like Ticket/Executor. This design is especially suitable for long-running Agents that need to receive new instructions repeatedly.
However, DeepSeek's public research prototype mainly targets code repository scenarios. Repo tasks usually have relatively clear scopes, file changes, and deliverables, while AI creation tasks have much fuzzier boundaries.
For example, a simple image generation may take only tens of seconds, but subsequent user modifications can gradually evolve it into a long-term creative project. Meanwhile, the same material may be referenced and modified by multiple tasks.
Therefore, we kept the basic Router/Worker division while adding designs for the creative scenario like lightweight tasks, shared artifact identities, and async waits.
This approach also introduces new complexity, such as task attribution judgment, shared object permissions, and cross-task state synchronization. We'll expand on these in later sections.
2.2 Not Every Request Needs an Independent Worker
Multi-agent architectures can easily fall into a trap: since splitting tasks is possible, why not hand everything to independent agents?
But independent execution spaces have their own costs. Every Worker created means maintaining extra context, runtime state, and message routing relationships, and model calls and coordination overhead may increase.
Take Lomo Agent's creative scenario as an example:
When the user asks "what resolutions does this video model support", Main can handle it directly;
When the user requests "generate an orange cat image", it may only need a single generation task;
But "make a set of style-consistent brand posters" involves planning, asset management, multiple rounds of generation, and subsequent modifications, which is more suitable for creating an independent task.
For this, we further distinguish execution tasks into Lightweight Task and Persistent Task.
Lightweight tasks carry single generations, execution states, and result cards, without occupying the user's task list long-term. When the user keeps modifying around the generation result, or the task gradually evolves into a complex creative goal, the system can upgrade it to a persistent task, preserving the context and execution records needed for subsequent collaboration.
Persistent tasks have a relatively complete lifecycle; they can continuously receive messages, execute plans, wait for external tasks, and resume work after days or even longer.
This division also complements the de-session design. We want to reduce the burden of user managing sessions, and naturally we don't want to just replace it with another task list that needs frequent tidying.
2.3 How a Collaboration Flows
Back to the brand poster and cat video example.
The user first initiates poster creation in Main; Router judges this is a goal that needs sustained advancement, so it creates a corresponding task, and Worker handles subsequent planning and generation.
During the poster production, the user asks to generate a video from the previous cat image. The system recognizes this as another independent job and creates a new execution space. The two tasks maintain separate contexts and can advance independently based on their states.
When the poster plan needs confirmation, the Agent presents candidate options via a task card. The user selects directly on the card, and the message can be sent back to the corresponding Worker based on the explicit task identity, without semantic judgment again.
A few days later, the user says on the main line "make that set of posters warmer", and Router, combining the task index and historical info, associates this new instruction with the original poster task.
There's a notable design detail here: message attribution and message execution are two different phases.
Router can assign a message to a task, but the Worker may not process it immediately. For example, it might be waiting for video generation or paused at a user confirmation node. New messages need to enter the persistent inbox first, and then the runtime decides when to consume them based on the current state.
This also means task organization is only the first step. As the number of tasks grows, how to accurately identify subsequent user intent, avoid misrouting, and handle new messages inserted during execution becomes a major issue for the whole architecture.
3. Message Routing: Let Every Message Find the Right Execution Unit
A unified interaction entry lets users switch tasks naturally, but it also hands a job previously done by the user to the system: deciding which unit each new message should go to.
For example, a user just generated an image and suddenly says: "Change that one to blue."
"that one" might refer to the latest generated image, or it might refer to material in a previous poster task.
If attribution judgment goes wrong, at best we create duplicate tasks, at worst the Agent modifies the wrong work.
So the accuracy of message routing directly affects the reliability of the whole system. We don't want all messages to rely on the model re-guessing attribution; we adopt a relatively simple principle: what can be determined by explicit relationships is handled directly by program; only when explicit attribution is missing does the model perform semantic judgment.
3.1 Deterministic Routing and Semantic Routing
Lomo Agent's message intake is roughly divided into three stages.
First layer: direct routing for clear goals.
When the user replies in a task panel, clicks a button on a task card, or initiates a modification against an explicitly bound artifact, the system already knows which goal the message belongs to and can deliver directly via Task ID or object relationship.
In this case, calling the model again to judge attribution adds latency and misjudgment risk.
Second layer: Router semantic judgment.
For messages sent from the main line without an explicit target, Router needs to combine the current request, recent interaction, and existing task index to judge.
Possible results:
- Simple requests handled directly by Main.
- Lightweight tasks needing only a single generation.
- Follow-up instructions that can be associated with an existing task.
- New goals requiring a persistent task.
- Clearly ambiguous attributions requiring user confirmation.
There's also an easily ignored issue here: Router doesn't need to read the full history of all tasks. It usually only needs the task's goal, summary, recent state, and relevant object references; after identifying candidate targets, it reads specific info on demand.
This avoids the routing itself becoming another ever-growing context as task numbers increase.
We currently prioritize providing recently active tasks, including index info like goal, summary, and state; if the user mentions earlier work, we retrieve via keyword and summary vector similarity, giving Router a small number of relevant candidates. This way, as task count grows, the main impact is retrieval scope, not the context length each inference must carry.
Another common error is treating work the user meant to continue as a new goal. To prevent this, before creating a persistent task, the system does a similarity check against the most recent 50 in-progress tasks. If it finds potentially duplicate goals, it hands the most similar candidates to the model to decide whether to reuse an existing task or confirm a new one. This check blocks some duplicate creations, but doesn't replace all semantic ambiguity clarification. When evidence is insufficient, the user must make an explicit choice.
Third layer: attribution and state check before execution.
After determining the message target, the system still needs to check whether the task is valid, whether it can receive the current operation, and whether the relevant object is still accessible.
For example, a user might reference an archived task or try to modify a deleted asset. Router giving the correct semantic attribution doesn't mean the subsequent operation can execute.
3.2 Message Delivery Does Not Mean Immediate Execution
One easily overlooked detail in message routing: the message being assigned to a Worker and the Worker actually starting to process it can happen at different times.
For example, while the Agent is generating a video, the user adds: "Change the background to indoor."
The video generation request may have already been submitted to an upstream service and cannot be modified immediately. The system needs to save this new instruction first, and only after the Worker reaches a safe execution boundary will it combine with the current progress to decide the next step.
Therefore, we maintain a persistent message inbox (Inbox) for each task. Router is responsible for delivering messages to the correct inbox, and the runtime is responsible for deciding when to read, how to merge into context, and whether to adjust the existing plan.
Several cases to distinguish:
- Messages not yet consumed by Worker can be retracted or replaced if conditions are met;
- Messages already in model inference need to wait for the current execution boundary;
- For already-submitted external generation requests, they cannot be directly retracted now. New instructions need to wait until the Worker reaches a safe execution boundary, then decide whether to regenerate based on task progress or apply the new constraints to subsequent steps.
This separation also provides the foundation for later interruption recovery, async tasks, and user intervention at any time.
3.3 Borrowing Message Attribution Design from Collaboration Products
In interaction design, we also referenced message organization methods from collaboration products like Slack and Feishu.
Take Thread replies: when a user replies under a specific topic, the system naturally knows which discussion the message belongs to, without re-analyzing the whole conversation for attribution. This explicit association is stable, predictable, and carries almost no semantic recognition cost.
However, Lomo Agent faces more flexible interaction scenarios. Users may reply directly on a task card, or return to the main line and casually mention a job from a few days ago.
So we keep both approaches: for explicit task operations, directly use structured identity routing; for natural language on the main line, Router associates with historical tasks.
This hybrid strategy reduces the number of times the model participates in routing, but cannot completely eliminate ambiguity. For example, if a user is making two poster sets simultaneously but only says "modify the last set", the system may not have enough evidence to determine the target. In this case, asking actively is usually more reliable than choosing what seems the most relevant task.
As task count grows, how to balance recall, routing accuracy, and interaction cost still needs to be continuously optimized through real usage.
However, attribution judgment is not the final check in the chain. Creating an extra task can usually be merged or closed; if a message is wrongly associated with artifact modification, asset deletion, or paid generation, the cost can be much higher. So we separate message routing from actual execution authorization: the former judges which task the work should go to, the latter still independently checks permissions and cost boundaries.
For example, irreversible operations like deleting assets or forgetting memories require user confirmation each time; paid generation gives a quote first, and requires confirmation when exceeding the no-confirmation quota, the daily no-confirmation limit, or the approved plan budget. Paid operations initiated via the open API are also confirmed one by one. Even if Router identifies the correct task, it doesn't gain execution rights beyond what the user authorized.
After the message enters the correct task, execution may still span minutes or days, encountering process restarts, network timeouts, or user cancellation. Reliable recovery is handled by persistent execution.
4. Persistent Execution: Interruption Recovery, Retry, and Concurrency Control
After messages are correctly routed to a task, the next problem is execution reliability.
For ordinary Chat Completion, a request usually completes in seconds to tens of seconds. But a creative Agent's workflow is much more complex: a single video production may involve multiple rounds of model inference, dozens of tool calls, and multiple external generation tasks that take minutes. During execution, the user may also modify requirements or actively stop the task at any time.
Consider a common failure scenario: the Agent submits a request to a video generation service; upstream has actually created the task successfully, but the process restarts right before receiving the response.
What to do after recovery? If we simply retry, we may generate duplicate content and charge again; if we consider the request successful, we may wait forever for a task that wasn't actually created.
Such problems made us realize that the Agent's execution process needs persistent state independent of the model context and the service process.
Therefore, I introduced a Durable Execution mechanism in Lomo Agent. The database records key execution state, model decisions, and tool call results; the task queue handles execution scheduling and recovery; real-time events give users progress feedback. Even if the execution process is interrupted, the system can judge the next step based on saved facts.
In design, we referenced Temporal's Durable Execution. Temporal lets business processes span individual process lifetimes through workflow history, task scheduling, and retry mechanisms, which is especially advantageous in complex, long-running workflows.
But general workflow engines have modeling and running costs too. Lomo Agent's execution process also needs to manage model inference, dynamic tool calls, creative state, and user interaction, so we implemented an execution ledger and event scheduling mechanism around the Agent's own data structures. We borrowed the engineering thinking of persistent execution, not directly using Temporal.
4.1 Persist Decisions Before Executing Tool Calls
The traditional Agent Loop follows a simple flow:
Read context → Model inference → Call tools → Get results → Continue inference.
For short tasks, this is straightforward. But when execution spans multiple processes, external services, and billing systems, any step's interruption can cause state inconsistency.
We divide the execution process into a series of Steps. Each Step can include one model inference and a set of tool calls proposed by the model.
Before executing tools, the runtime first persists the model's decision and the pending calls. Each tool call has a stable business identifier, and its status, input, and result are recorded in the execution ledger.
This way, when a Worker recovers, it can determine from the ledger which operations are done, which can be retried, and which need to verify external state first, without re-invoking the model to regenerate the same round of tool decisions.
A key concept here is Idempotency.
Suppose a user approves generating a video, and the system resubmits the same request due to a network timeout. Ideally, the two submissions should correspond to a single generation order, not produce two duplicate videos, and not double-charge.
For downstream services supporting idempotency keys, we can use a stable call identifier for deduplication; for the platform's internal order and billing processes, we can use call ID, order ID, and unique constraints to ensure duplicate requests produce no new business effect.
But if the upstream generation interface supports neither idempotency keys nor result querying, then after a network timeout, the system cannot actually confirm whether the task was created.
In this case, persistent execution cannot magically provide Exactly-once guarantees.
We need to further distinguish two situations: for the few non-idempotent tools, if the ledger records started but never gets a result, the runtime will not directly re-execute; instead, it gives the model a result marked outcome_unknown, and lets the model first check the current state before deciding subsequent actions. For idempotent calls, we can replay with the original call identifier.
Another layer of uncertainty occurs between the generation service and the upstream interface. Here we take a more conservative strategy: if the submission result is unknown, we judge the submission as failed and do not automatically re-initiate the request to upstream. Both seem to be "result unknown", but one is handling recovery of Agent tool calls, the other is avoiding duplicate creation of actually charged generation tasks.
Temporal faces the same issue in Activity retries. A workflow engine can reliably schedule and retry executions, but external API side effects still require business idempotency mechanisms.
For paid generation services, this boundary is especially important. An execution failure can be recovered, but duplicate charges may not be automatically compensated.
4.2 Worker Takeover and Fencing Token
Persistent execution solves recovery after process interruption, but there's a more subtle situation: the original Worker may not have actually exited.
For example, Worker A loses connection during execution due to network jitter. The system detects a timeout, and Worker B takes over the task and continues. A few seconds later, Worker A reconnects and tries to submit its just-completed results.
At this point, the system might have two Workers that both believe they own the execution. Without restriction, the old Worker can overwrite new state, or even bring an already-stopped task back into execution.
To address this, we use an execution generation mechanism similar to Fencing Token.
Every time a new executor takes over a task, the runtime increments the task's execution generation. Workers must carry and validate their generation when submitting state changes.
If the current generation in the database has been updated, the old Worker's submission is rejected.
This mechanism can be understood as: the system allows the new executor to take over work while revoking the old executor's right to continue modifying internal state.
When a user actively stops a task, we also need to check running state and execution generation to prevent the old Worker from continuing after recovery.
However, Fencing Token protects the system's own state consistency. If the old Worker has already initiated a video generation request to an external platform, revoking internal execution rights doesn't automatically cancel the remote operation.
Therefore, even if the user stops the task, a previously submitted external generation may continue. The generation service will keep tracking the task and complete billing settlement, but the late completion event won't reawaken a stopped task or change an already-cancelled execution state.
4.3 Handling New User Messages During Execution
Another class of problems comes from real-time user intervention.
Suppose the Agent is calling the model to generate a set of posters. At this moment the user sends: "Background should be dark blue, not white."
This message may have arrived at the server before the model inference ends, but the current model request hasn't actually read it.
If the system only concatenates history by message creation time, state confusion can occur: the model appears to have made a decision after the user's modification request, but actually it never learned about the new instruction.
To solve this, we separate message reception from message consumption.
New messages first enter the corresponding task's persistent Inbox. The Worker reads these inputs at the next safe execution boundary and organizes them into the model context along with current task state.
The system records at which Step a message was actually consumed, not just relying on send time to determine order.
This clearly distinguishes the stages of user instruction delivered, model understood, and operation executed.
Of course, if a generation request has already been submitted upstream, later modifications won't automatically change the content being generated. The current version cannot retract such requests; the Worker can only decide at the next safe execution boundary whether to regenerate or apply the new constraints to subsequent steps. If the user has stopped the task, even if old generation completes later, it won't restart the task.
From the user's perspective, the interface must accurately reflect this state, avoiding the misreading of "message received" as "current generation already adjusted to new requirements."
For time-consuming generation tasks like video, audio, and 3D, the Worker also needs to release execution resources while waiting for external results and accurately resume when the event arrives.
4.4 Execution Ledger and Scheduling Consistency Boundary
We've discussed persisting model decisions and using execution generation to avoid old Workers overwriting state after takeover. But in actual operation, there's an easily overlooked problem: if the database has recorded work to be executed but the queue message that wakes the Worker is lost, how does the task recover?
In Lomo Agent, we maintain execution state through Turns and tool-call ledgers. A Turn records running state, execution generation, lease, and checkpoint. Tool calls use "turn + step number + call sequence" as stable identifiers. After the model returns, the runtime saves the decision, pending calls, and consumed Inbox messages in the same transaction; before tool execution, it records started, and after completion, saves results and done state together. This way, on recovery, we can reuse results or retry safely based on existing records, without making the model re-infer what happened earlier. For calls that cannot be safely retried, we keep the unknown-result state for subsequent decisions.
For task scheduling, we did not introduce a transactional Outbox for turn enqueueing. Instead, we treat the database as the source of truth for execution facts, with the queue mainly responsible for waking Workers. After the turn commits, we send a scheduling message; if enqueue fails, a background watchdog rediscovers the task based on database state. Currently, the watchdog checks every 10 seconds, focusing on expired leases, long queue times, and tasks with unprocessed messages but no active turn. Queue tasks are identified by turn ID and execution generation to reduce duplicate enqueues of the same generation; even if duplicate scheduling occurs, the runtime still validates execution state and generation to prevent stale writes. This means we accept limited recovery latency in exchange for a simpler scheduling mechanism; UI event delivery and external generation tasks each retain their own reliable delivery and compensation mechanisms.
Another class of risk comes from concurrent execution of old and new Workers. Fencing Token can prevent the old Worker from overwriting internal state, but cannot revoke external requests already sent. For this, we fix the paid generation idempotency key as "turn + step number + call sequence" without including execution generation. Even if old and new Workers resubmit the same call, the platform generation layer associates it with the same order via the unique constraint "workspace + idempotency key", and deduplicates internal billing. This controls internal duplicate orders and billing, but does not mean external upstream has an Exactly-once guarantee. In contrast, if a fault occurs before the model decision is persisted, the step may be re-inferred. Given the current cost structure dominated by media generation, we choose to accept this relatively limited duplicate-inference cost.
A more difficult situation is when upstream submission results are unclear. A request timeout could mean submission failure, or upstream created the task but the response didn't return. To avoid duplicate generation, the current generation service does not automatically resubmit such requests; once a successful upstream task ID is obtained, it is persisted immediately to allow continued polling after interruption. But if the process crashes between obtaining the ID and saving it, we currently cannot look up that task. The system can only process refunds per rules after timeout, with the platform bearing any upstream costs already incurred.
These designs cannot eliminate all fault windows, but they clarify the handling boundary for different situations: what can be confirmed as completed is directly reused; what can be safely retried is continued; what cannot be confirmed avoids blind duplicate submission. For a continuously running Agent, reliability doesn't mean every step never fails, but that after interruption the system can make controllable recovery decisions based on existing facts.
5. Async Generation: Task Lifecycle and Event Wake-up
The persistent execution introduced above mainly solves how the Agent records state, handles faults, and recovers during operation. But in AI creation scenarios, there's another unavoidable feature: a large number of tool calls are themselves asynchronous.
For example, image generation may take seconds, but video and 3D tasks may last minutes or longer. After submission, the upstream service usually returns only a task ID; the actual generation result is obtained through polling or callbacks.
If Workers wait idly for external tasks to complete, they not only occupy execution resources but also couple model inference with uncontrollable external generation time.
Therefore, we manage the Agent's execution lifecycle separately from the external generation task lifecycle.
5.1 Submission, Waiting, and Resumption
When a Worker calls a generation tool, the tool first submits the request to the corresponding service and returns the generation task identifier. The runtime then saves the association between the task and the current matter, including task ID, waiting state, and info needed for subsequent execution.
After submission, the Worker can suspend its current execution without continuously calling the model or maintaining a long-running inference request.
During the wait, progress changes from the generation service can be pushed to the frontend directly via business events, and the task card displays current status.
When the task completes, the runtime decides whether to reawaken the Worker based on whether subsequent steps depend on the generation result.
The direct benefit of this design is improved resource utilization. While waiting several minutes for video generation, the system only needs to maintain persistent state and necessary event processing, without occupying Agent inference resources.
Also, generation progress and the user interface can update independently. Users see task status in real time without waiting for the Worker to call the model again.
5.2 Async Events Also Need Reliable Handling
After switching generation tasks to event-driven, we need to consider event reliability itself.
For example, upstream may send duplicate completion notifications due to network issues, or a callback may fail and not be retryable immediately. For multiple concurrent generation tasks, progress events may arrive out of order.
So event handling cannot simply mean "continue execution upon receiving a callback."
We need to update the generation task state based on the persistent task state, ensuring duplicate events don't cause duplicate writes, duplicate billing, or duplicate Worker wake-ups.
For events that may arrive out of order, we also need to avoid old progress overwriting final results. For example, if a task is already complete and then a delayed "generating" notification arrives, the state should not regress.
Additionally, external callbacks are not necessarily reliable. For tasks with no results for a long time, we can check status against the upstream query interface using the task ID, and use compensation polling to restore task progress when necessary.
This works together with the Durable Execution described earlier: the runtime maintains trusted execution state, and the async task system converts external generation changes into recoverable business events.
5.3 When Does the Worker Need to Be Woken?
There's another question directly related to model call cost: after a generation task completes, do we always need to call the model again?
The answer depends on whether there is Agent-decision work remaining.
For example, the user only asks to "generate a cat image." After completion, the system can display the result directly; there's no need to wake the Worker and make the model re-describe the generated image.
But if the user asks to "make a set of style-consistent brand posters", the current generation result may be an input to subsequent steps. The Agent needs to check existing artifacts, read the execution checklist, and decide whether to continue generating or wait for user confirmation.
Therefore, we separate generation-completion events from Agent continuation decisions.
The generation service reports results; the runtime decides whether to resume the Worker based on the task plan, waiting relationships, authorization, and current execution state.
This also avoids handing a flood of progress events to the model. For a video task lasting minutes, upstream may produce many state changes, but most only need to update the frontend, not trigger new inference.
This event-driven design is similar to async wait mechanisms in general workflow engines and task queues. For Lomo Agent, we further focus on combining it with creative goals, execution checklists, and user review flows.
For example, after a video shot generation completes, if other shots remain and the user has authorized continued execution, the runtime can resume the corresponding task; if the current stage requires user review, it should stay in a waiting-for-confirmation state.
Generation completion only means a tool call has ended. Whether the entire creative task should continue still requires judgment based on plan, budget, and authorization.
This leads to the later autonomous progression mechanism: when the Agent finishes a stage, or even prepares to end a reasoning turn, how does the system decide whether the task is truly complete and whether to continue?
6. Context Engineering: Maintaining Valid State in a Limited Window
As the Agent handles long, multi-stage creative tasks, context management becomes a key factor affecting execution quality and cost.
A creative matter that lasts days may accumulate hundreds of messages, dozens of tool calls, and a large amount of image, video, and workflow state. Even as model context windows grow, stuffing all of it into every inference is unrealistic.
On one hand, long context brings higher token cost and inference latency; on the other, excessive history can interfere with the model's judgment of the current goal. Especially when users modify requirements repeatedly, early invalid constraints mixed into context can easily affect subsequent generation.
So we treat context management as an independent engineering problem: at each inference step, how to give the model sufficient, relevant, and still-valid information.
6.1 Context Layering and Prompt Cache
Lomo Agent organizes model context into several layers: a relatively stable system prefix, compressed history, recent interactions, and the dynamic state needed for the current execution step.
A factor closely tied to real inference cost is Prompt Caching. Hereafter, Prompt Cache refers to the prompt caching capability provided by model service providers at the API layer; its specific implementation and reusable range are determined by the provider.
Model services that support this capability can reuse prompt prefixes matching their caching rules, reducing repeated processing costs. Therefore, we try to keep system rules, core tools, and relatively stable profile information at the front of the context, and put task state that changes each step at the tail.
If highly dynamic information is inserted at the beginning of the prefix, subsequent stable content may not effectively reuse the cache.
However, actual Prompt Cache hit rate is also affected by model provider, request routing, and caching strategy. Context layout only creates conditions for cache reuse; final benefits still need validation through actual call data.
Also, a stable prefix doesn't mean profiles can stay outdated. When users modify preferences or ask to forget certain information, the relevant profile snapshots must be invalidated promptly, avoiding stale info participating in inference due to cache reuse.
6.2 Tools Loaded On Demand; More Capabilities Mean More Expensive Context
Lomote AI integrates many creative models and tools, covering image generation, video generation, 3D, asset management, document editing, and Flows workflows.
If we put all tool names, parameters, and usage instructions into every model request, we not only increase token consumption but also risk the model making wrong choices among many similar tools.
So we divide tools into two categories: core tools that need long-term visibility, and tools dynamically loaded by capability domain.
For example, when the Agent is doing storyboard planning, it may only need document and asset tools; when entering video production, load video generation and workflow capabilities.
For tool areas no longer active, we can remove detailed definitions at context compression boundaries, reducing irrelevant info's impact on subsequent inference.
This design has costs. When the tool set changes, the model context structure may change, affecting Prompt Cache hit rate; historical tool calls may be inconsistent with currently visible tool definitions.
On this point, Manus's Context Engineering practice gave us a thought-provoking counter-example.
Manus tends to keep tool definitions stable, reducing tool-selection complexity by constraining current available actions. This helps reduce prefix invalidation from tool definition changes and makes it easier to maintain consistency between historical tool calls and current context. The discussion here focuses on the impact of stable context on cache reuse, not equating API-level Prompt Caching with KV Cache inside model inference.
Lomo Agent chooses domain-based loading mainly because the platform's tool types and parameter structures are large, and long-term retention of all tool definitions may incur high context overhead.
Both strategies have applicability: stable tool sets favor caching and behavioral consistency; dynamic loading may reduce token cost and selection interference from irrelevant tools.
For our approach, the actual effect needs continuous evaluation by tool switching frequency, cache hit rate, tool call accuracy, and overall inference cost. Additionally, whether tools appear in the model context only affects what the model can choose now. Actual tool execution permissions are still controlled server-side; tool visibility in the prompt cannot be relied on as a security boundary.
6.3 History Compression: Beyond Summaries, Keep Read-back Ability
During long-term execution, history messages and tool results keep growing. Even with a large context window, compression is needed at appropriate stages.
A common approach is to have the model continuously summarize previous history and continue the conversation using the latest summary. But this has a problem: summaries may gradually lose details through repeated compression, and even solidify the model's own guesses as historical facts.
Therefore, we adopt segmented summaries, compressing earlier history into intervals while retaining recent interactions in full.
New summaries are appended to the existing summary sequence when possible, avoiding rewriting the whole history each time.
For large tool outputs, such as asset lists, workflow node info, or complex generation results, the system can retain only a necessary summary in the current context and store full content externally.
When the Agent later needs to check a specific parameter or historical detail, it reads it back through retrieval or pagination.
This is similar to Manus's approach of using a file system as an external context carrier. The model doesn't need to carry all info long-term, but must be able to re-find reliable original evidence when needed.
A notable difference here: summaries are to reduce the context burden of current inference, and should not be the only source of historical facts. Especially when a user corrects information or a historical constraint becomes invalid, the system needs to locate the relevant source to avoid old summaries influencing subsequent execution.
6.4 Keep the Current Task Always in the Model's View
Context compression solves the problem of overlong history, but another equally important problem is: when the Agent is halfway through, does it still know what goal it's working toward?
For long-running tasks, the model may drift from the initial goal after multiple tool calls. For example, the user asks for a set of brand posters, but the Agent spends excessive effort on optimizing one reference image, ignoring the overall deliverable.
Therefore, we continuously provide the Worker's dynamic context with the current task brief, execution checklist, key constraints, waiting state, and latest changes to relevant artifacts.
These states come from the persistent runtime, not just a model summary of previous chat content.
For example:
- The task brief records the visual direction the user finally approved.
- The execution checklist records which steps are done and which remain.
- Workspace state tells the model whether a material was replaced by the user and which version the current document is at.
When the user directly modifies a Flows workflow or shared document, the next execution round can receive an object-change notification and read the latest content when needed.
This way, even if some history is compressed, the Agent can continue based on trusted task state.
Manus also inspired us by continually presenting a todo list to maintain the model's focus on the goal. Lomo Agent further combines this with runtime state, making it both a model-context input and a basis for auto-continuation and stop decisions. This is detailed in the autonomous progression section.
6.5 Practical Trade-offs in Context Engineering
In sum, Lomo Agent's context engineering targets three goals:
- Reduce irrelevant information
- Control inference cost
- Ensure the information used for current decisions is still valid
But these goals aren't always consistent.
More aggressive compression reduces token consumption but may omit important details. Dynamic tool loading reduces the tool-choice space but may affect cache hits. Long-term stable profiles enhance continuity but must handle preference changes and memory revocation.
So context management can't rely on one fixed strategy to solve everything. It needs to be adjusted together with task state, memory system, tool permissions, and actual model behavior.
For Lomo Agent, we care more about whether the model can find the right info when needed and continue based on currently valid task state. When collaboration spans multiple matters, multiple days, or even months, context management within a single matter is not enough. Long-term user preferences, historical projects, and past decisions need to be organized and retrieved by another mechanism.
7. Memory Engineering: Sources, Versions, and Revocability
The previous chapter discussed how to organize information in a limited context window. But for a long-term collaborative Agent, there's a deeper question: when a conversation has left the current context, or even when the related matter has ended, what should the system remember?
In Lomo Agent's creative scenarios, users may use the platform for months, gradually forming relatively stable visual preferences, common creative flows, and brand guidelines. These shouldn't require the user to re-describe them each time.
At the same time, user preferences change. A temporary style used in one project may not apply to all future creations; a brand guideline that held previously may become entirely invalid after a rebrand.
Therefore, we want long-term memory to provide continuity across matters and time while preserving the source and validity scope of facts.
7.1 From History Messages to Long-term Memory
Long-term memory first needs to solve information filtering.
For example, when a user says "I usually like low-saturation color palettes", this may be a preference worth keeping long-term; but if they say "let's try high-saturation red this time", it's more likely a temporary requirement for the current work.
Both sentences involve color but have different scopes of applicability. If the system directly writes all user messages into a vector database and relies on semantic retrieval to fill context, it will easily confuse long-term preferences, temporary requirements, and invalid historical info.
To this end, we divide Lomo Agent's memory into several layers:
| Memory Layer | Content Saved | Main Purpose |
|---|---|---|
| Evidence | User messages, sources, time, and validity status | Basis for fact extraction, correction, and revocation |
| Atomic Memory | User preferences, project facts, common habits; records source and version | On-demand retrieval across matters |
| Episodic Memory | Stage summaries of a conversation or matter | Helps Agent recall historical projects and progress |
| User Profile | A small set of relatively stable, high-value info | Provides low-cost background for daily collaboration |
These layers have associations but different responsibilities.
Original messages are traceable evidence. The system can extract candidate memories from them, then decide whether to save, supplement, or update based on source, time, scope of application, and existing records.
Episodic memory focuses on what happened in a collaboration. For example, last week the user made a set of brand posters, eventually chose a warm color scheme, and decided to postpone video production. This info may not all deserve to enter the long-term profile, but could matter next time the user mentions the project.
The user profile needs to be more restrained. It should keep a small set of relatively stable info, avoiding becoming another ever-expanding history conversation over time.
During actual recall, the system chooses the memory to provide to the model based on the current matter's goals and permissions, without loading the full history every round.
An important principle here: memory must distinguish facts clearly expressed by the user from the model's own speculation.
For example, because the Agent generates several cyberpunk images in a row, it infers the user has a long-term preference for cyberpunk style. This can be a candidate judgment, but cannot be equated with a preference the user explicitly expressed.
Similarly, webpage content, tool results, and the Agent's own generated summaries cannot be repackaged as user personal facts without reliable basis.
7.2 Inspiration from Existing Memory Architectures
In designing long-term memory, we referenced several technical routes.
OpenClaw's memory practice emphasizes inspectable, maintainable storage and the relationship between memory and original context. Its advantage is high transparency, letting users or developers inspect what is saved; but as memory scale grows, retrieval, maintenance, and conflict handling still need extra mechanisms.
Mem0 focuses on extracting valuable info from conversations and recovering relevant memories through retrieval in later interactions. This suits long-term, cross-topic agents, but memory quality highly depends on extraction accuracy and recall strategy. If temporary instructions are misjudged as long-term preferences, subsequent retrieval may continuously amplify the error.
Zep / Graphiti offers another approach: explicitly maintain entity relationships and temporal information in memory. This is especially valuable for handling changing facts. For example, a brand previously used blue as primary color, then changed to green; the two records don't necessarily conflict if they correspond to different valid times.
However, temporal relations and knowledge graphs introduce higher maintenance complexity: entity normalization, relation updates, conflict handling, and retrieval cost.
Combined with Lomo Agent's creative scenario, we focus on three questions:
- Whether user preferences have reliable sources
- Whether historical facts remain valid
- Whether related memories can be updated synchronously after user correction or revocation
Therefore, beyond common layered storage and semantic retrieval, we also maintain source, version, and derivation relationships for memory.
7.3 Why Forgetting Is Harder Than Remembering
Long-term memory also has an easily overlooked problem: when the user explicitly asks to forget certain content, how should the system handle it?
Suppose the user initially says: "Our brand's primary color is blue."
Later they add: "Blue with off-white gives a softer overall feel."
The memory system might merge the two expressions into a new brand preference: "This brand prefers blue with off-white, and tends toward a soft visual style."
If the user later asks to forget the initial blue setting, deleting only the first atomic memory is clearly insufficient, because the merged version still contains related info.
As the system runs, this info may also enter episodic summaries, user profiles, and even be merged into other derived memories.
Therefore, we maintain source and derivation relationships for memory, describing which historical evidence and old versions a memory evolved from.
In actual implementation, we maintain the evolution of memory through a version chain. Taking the brand primary color example: the initial blue expression forms valid version v1, linked to original evidence (ORIGIN); when the user adds the off-white matching requirement, the system produces v2, marking the original version as SUPERSEDED and recording new correction evidence (REFINE). If the user merely confirms an existing conclusion, it can be recorded as CONFIRM without re-creating a memory.
When the user asks to forget the initial blue setting, the system revokes usage eligibility of the corresponding evidence and locates affected memories along the version relationship. Versions relying on that evidence are deleted and their content cleared, then re-extracted based on remaining valid evidence; evidence that only served to confirm is simply disassociated, leaving independently valid memories intact.
This revocation process also propagates to the user profile and episodic summaries. When a related profile entry becomes invalid, the system updates the profile version so subsequent inference no longer uses the old snapshot; affected summaries need to be rebuilt. At the same time, the system deletes corresponding retrieval vectors and suppresses re-extraction of deleted content, reducing the chance of old info re-entering context.
Note that already-sent model requests cannot be instantly revoked. To this end, we also track the messages and memories that replies and tool results were based on, and exclude affected derived content in subsequent inference, preventing forgotten info from being reintroduced through the Agent's own historical outputs.
The "forgetting" discussed here mainly targets usage eligibility of long-term memory and derived context, and does not equal deleting all original chat records, generated materials, or transaction data. That is a different data lifecycle and privacy management issue and needs separate handling.
7.4 Memory Reliability and Degradation Strategy
Source tracking and version management increase memory system controllability, but do not completely eliminate model understanding errors.
For example, a single conversation may contain multiple candidate facts, one meeting save conditions, another being a momentarily misunderstood instruction. If we treat the whole batch of extracted results as an inseparable unit, a single anomaly can affect other valid memories.
Therefore, candidate memories need per-item verification and handling, distinguishing cases like unsupported source, content conflict, and output format abnormality.
When the memory service fails, normal creative tasks should not come to a halt.
For example, retrieval timeout can temporarily skip memory supplement; background extraction failure can be reprocessed later; a single candidate format anomaly doesn't have to cause other valid records to be lost.
This degradation strategy means the Agent may sometimes lack complete historical preferences but can still continue based on the current task brief and existing execution state.
Compared with letting a problematic memory result affect the whole task, we prefer to keep the normal execution path and provide relevant background again after memory recovers.
The value of long-term memory ultimately needs validation through actual collaboration. Besides recall accuracy, we also care whether memory is stale, whether sources can be explained, and whether old info still affects subsequent decisions after user correction or revocation.
With memory and task context gradually improving, the Agent can have cross-time background info. But knowing what the user did in the past doesn't mean it can judge whether the current task should continue.
A matter with multiple creative steps needs an explicit execution plan, stop conditions, and user authorization boundaries. This leads to the next chapter's autonomous progression mechanism.
8. Autonomous Progression: Planning, Stop Conditions, and User Authorization
The mechanisms above let the Agent maintain long-term tasks, recover interrupted execution, and retrieve historical info when needed. But for a creative goal that needs continuous advancement, one question remains unsolved: how does the Agent judge whether it should keep working?
For example, the user asks to make a brand short film. The Agent completes the script and storyboard, then outputs a summary and ends the current turn. The model hasn't made a mistake; it really finished the current stage, but there's still a long way to go before the final film the user wants.
In early demos and scenario drills, we encountered similar situations. Users had to repeatedly send "continue" before the Agent would move to the next stage.
On the other hand, if we simply ask the model "don't stop until all tasks are done", another problem arises: the Agent may repeatedly try already-failed operations, generate unnecessary material, or expand creative scope without user confirmation. So we need task goals and execution progress to be independent of a single model reply, and let the runtime control the boundaries of autonomous advancement.
8.1 Turning Plans into a Maintainable Execution Checklist
We introduced an Execution Checklist to record the main steps from plan to delivery for a creative goal.
Unlike a normal TodoList the model generates in chat, the execution checklist is a structured state persistently maintained by the system. Each step can associate execution status, artifacts, completion evidence, and necessary user confirmation requirements.
For example, making a brand short film can be organized as:
| Execution Stage | Main Task | Completion Basis |
|---|---|---|
| Script & Storyboard | Determine narrative and shot design | Storyboard doc and user confirmation |
| Character & Scene | Prepare visual reference material | Corresponding assets and version records |
| Shot Generation | Call video model to generate shots | External generation tasks and results |
| Plan Review | Check shots and overall direction | User feedback or authorized acceptance results |
| Final Delivery | Organize deliverable artifacts | Final work and associated assets |
Once the user approves the execution plan, the checklist becomes the basis for subsequent advancement. The Worker can plan and call tools based on the current step, and the runtime can check progress after each execution round.
Note that the execution checklist is not a fixed, unchangeable plan.
Creative processes often change. For example, during shot generation the user decides to change the character design; the original steps and some artifacts may need adjustment. The system needs to update the plan based on new user instructions and maintain the association between steps and artifacts.
The execution checklist solves the persistence of goals and progress. It lets the Agent find its execution position after context compression, async waits, or process recovery.
8.2 After the Model Stops Outputting, Should the Task Continue?
Once the plan is persisted, we need to answer a more specific question: when the model completes the current turn, how should the runtime handle it?
In designing this, we referenced Codex Goals and Claude Code Hooks.
Codex Goals combines goals with continuous execution state, letting the Agent continue around unfinished goals; Claude Code's Stop Hook allows extra checks when the model is about to stop, and can decide whether to block the stop.
These designs inspire me: a model ending a reply doesn't necessarily mean the whole task is complete. The runtime needs an independent goal state and stop-check mechanism.
However, the completion standards for code agents and creative agents differ significantly.
In code tasks, tests passing, build success, or expected file changes provide relatively clear execution evidence. Creative tasks lack a unified objective standard. A video successfully generating doesn't prove character consistency, shot rhythm, or overall visual effect meets user expectations.
Therefore, we separate process progress from artifact acceptance.
The former can be judged by the execution checklist, tool call records, and artifact states; the latter needs multimodal understanding, specific task requirements, and necessary human review.
The runtime checks after each execution round whether the current task has pending steps, external waits, user confirmations needed, or reached execution limits.
We need to pay special attention to Meaningful Progress.
For example, adding a usable creative artifact, completing a checklist step, or successfully submitting a new generation task can all be references for execution progress.
But if the Agent is only repeatedly revising plan descriptions or toggling step status between "pending" and "in progress" without producing a new valid result, it shouldn't be allowed to continue indefinitely.
We use progress checks and execution limits to reduce the inference cost of such empty loops as much as possible.
Of course, this judgment still has limits. Generating new material only proves the system performed an operation, not that the material meets quality standards. More complete multimodal acceptance remains a direction for future exploration.
8.3 Human-in-the-Loop: Let Users Participate in Key Decisions
In creative work, users often want to delegate repetitive operations to the Agent while retaining control over key creative directions.
For example, brand visual direction, character design, storyboard plans, and final shot selection are usually highly subjective. If the Agent decides these nodes itself and continues generating, substantial invalid costs may follow.
Therefore, we introduce Human-in-the-Loop Checkpoint, pausing automatic advancement at stages that need user participation.
When reaching a checkpoint, the Agent can present the current plan, candidate results, and suggested actions via a structured card. The user directly chooses "confirm continue" or "adjust plan", and the system updates task state based on the explicit operation result.
This interaction style also references permission-confirmation methods in products like Claude Code: for operations with obvious side effects, the system should inform the user in advance and let them decide whether to continue.
However, human checkpoints in creative contexts involve not just permissions but aesthetics and creative direction. The user may have authorized the Agent to generate video yet still wants to personally choose among shot designs. Therefore, checkpoints must be saved as independent execution states, not rely solely on the model saying "waiting for confirmation" in its reply.
In development drills, we encountered exactly this: the Agent had reached a stage requiring user confirmation, but only replied "please confirm whether to continue" without providing an actionable confirmation card, so the user had to type "continue" again.
For this, we added a fallback mechanism in the runtime: when a task is paused at a checkpoint but the model didn't issue the corresponding prompt, the system automatically supplies a confirmation card; for generation executed only after user approval, if the execution checklist still has pending steps, the runtime also automatically wakes the task after generation completes. Thus, waiting for confirmation and resuming execution no longer rely entirely on the model's interaction behavior, but are coordinated by the runtime based on task state.
Of course, checkpoints don't mean every step needs user involvement. For irreversible or out-of-scope operations, explicit confirmation is always required; repetitive, low-risk steps can advance continuously within the authorized scope, avoiding excessive confirmations breaking the creative flow.
8.4 Autonomy Level, Budget, and Authorization Boundaries
In an AI creation platform, the Agent can call text, image, video, audio, and 3D generation capabilities from different model vendors, often with real costs. Therefore, autonomy level and budget control need separate designs.
The user allowing the Agent to execute automatically does not mean agreeing to unlimited calls to paid models; similarly, approving a budget doesn't mean all operations can skip review.
We maintain three dimensions in the runtime:
| Control Dimension | Problem Solved | Typical Mechanism |
|---|---|---|
| Autonomy Level | How far the Agent can advance continuously | Confirm each step, confirm at key nodes, auto-execute within authorization |
| Budget Control | How many resources the current task may consume | Per-call quota, plan budget, balance check, confirmation for additional budget |
| Execution Scope | What work the Agent is allowed to complete | Approved brief, plan, and explicit user modifications |
For example, the user approves generating three videos and allows the Agent to complete them automatically within budget.
If execution reveals the need for an additional character reference image, the Agent can judge whether this operation falls within the original plan and budget. If it needs to expand execution scope or increase costs, it should request user confirmation again.
An important security boundary here: the plan proposed by the model and the authorization claimed by the model cannot themselves become real user authorization.
The system should judge based on explicit user operations in the interface and authorization state saved server-side.
For different tools, we also adopt different control strategies based on actual side effects.
- Read-only queries are usually low risk
- Document writes need version and permission handling
- Paid generation needs budget checks
- Deleting assets or revoking memories are irreversible and require stricter confirmation mechanisms
Compared to coding agents, creative-platform agents have a special constraint: many external generation operations incur costs once submitted, and even if the user stops the task afterward, full revocation may not be possible. Therefore, budget and authorization checks need to happen before the side effect truly occurs, combined with the persistent execution mechanism described earlier.
From current practice, the difficulty of autonomous execution also lies in judging when to stop. When encountering insufficient balance, user pause, external task waits, necessary human review, or continuous execution without meaningful progress, the runtime needs to preserve current state and wait for conditions to be met before resuming.
This also means higher autonomy is not always better. Different creative tasks have significant differences in risk, cost, and verifiability. The Agent should minimize unnecessary intervention within the user's authorized scope, while retaining user control at key nodes.
Longer term, we hope multimodal quality assessment can let the system more accurately judge character consistency, shot coherence, and visual effect. As automatic acceptance improves, some stages that currently need human confirmation may be handled autonomously.
Of course, continuous collaboration also involves a problem beyond task execution: users may modify works directly in the workspace instead of through dialogue. When the Agent and user edit the same document or workflow at the same time, how does the system maintain state consistency?
This requires the shared workspace to maintain joint access and modification of works by both humans and agents outside task execution.
9. Shared Workspace: Let the User and Agent Modify the Same Work
The mechanisms above address how the Agent understands tasks, maintains context, and continues work. But in actual creation, collaboration between user and Agent isn't limited to the chat window.
For example, while the Agent is generating video from a storyboard, the user might directly adjust a shot's prompt in the right-side workspace, replace reference material, or modify model nodes in a Flows workflow.
These operations don't go through dialogue, yet they materially change the work. If the Agent continues based on previous context, it may use stale parameters or even overwrite modifications the user just made.
Therefore, in Lomo Agent we treat creative artifacts as persistent objects independent of dialogue and task context, organizing human-agent collaboration through the Shared Workspace.
9.1 Let Works Have an Independent Lifecycle
In traditional dialogue-based Agents, generation results are often attached to a chat message. Users can view, download, or continue asking, but as conversation grows, artifact management and subsequent modification become constrained by conversation structure.
Lomo's creative flow involves many long-maintained objects: images, videos, 3D assets, storyboard documents, asset collections, and Flows visual workflows.
These objects need stable identities, versions, and access permissions; they can be directly edited by users and referenced across different tasks.
So we manage dialogue context and creative artifacts separately:
- Dialogue and task context maintain user intent, execution process, and info currently held by the Agent.
- Shared workspace saves the works that truly need continuous editing and delivery, including object identity, content, versions, and related references.
For example, the user initially discusses a brand film storyboard in Main, and later a video-production task continues. Both execution spaces can reference the same storyboard document within permissions, without each creating its own copy.
Similarly, when a video-generation task ends, its artifact can still be referenced by new workflows, independent of whether the original Worker still runs.
This design lets works span the lifecycle of a single conversation or task, and lays the foundation for material reuse and multi-task collaboration.
9.2 Let the Agent Perceive Changes in the Workspace
Shared objects solve how works are saved, but also need to solve how the Agent knows a work was modified.
Suppose the Agent reads a storyboard document and prepares to continue generating based on its shot descriptions. At this moment, the user directly modifies the second shot in the workspace.
If the system only records chat messages, the Agent's next inference won't know about this change.
Therefore, we maintain version information for shared objects and record the version that the relevant task last read or modified.
When a document, asset, or workflow changes, the system can leave a persistent change hint for associated tasks. For example:
The storyboard document has been updated from v3 to v4; the second shot's description changed. Please read the latest version before continuing.
Here we have not chosen to trigger Agent inference on every workspace edit.
The reason is simple: when adjusting a workflow, the user may drag nodes, change parameters, or replace assets continuously. If every change wakes the Worker, we'd produce a flood of meaningless model calls and the Agent might intervene repeatedly while the user is still editing.
Therefore, change hints mainly let the Agent discover its info is stale on its next execution. Only when the relevant task indeed needs to continue does the runtime decide whether to resume execution based on current state.
For visual workflows like Flows, the Agent doesn't need to read full node data in every inference round. The system can provide brief change info first, and when the model actually needs to handle relevant nodes, fetch the latest content on demand.
This approach also connects with the earlier context engineering: the workspace stores the full artifact state, while context only provides what the current decision needs.
9.3 Version Conflicts and Optimistic Concurrency Control
Being able to perceive changes isn't enough to guarantee safety under concurrent modification.
Consider a more specific scenario:
The Agent reads storyboard document v3, ready to adjust the third shot. Meanwhile, the user directly edits the same document and saves it as v4.
If the Agent then submits a modification based on v3, it may overwrite content the user just put into v4.
Therefore, we use Optimistic Concurrency Control.
The Agent obtains the current version when reading an object and carries that version when submitting modifications. The server checks whether the object is still at the expected version before writing.
If versions match, the modification commits normally; if the object has been updated, the system rejects the operation that may overwrite new content and requires the Agent to re-read the latest state.
However, different creative objects have different granularity for version conflict detection.
For documents, a full rewrite must carry the version it was based on. If the user has modified the document, the system rejects the overwrite; partial modifications are located via precise, unique text spans, and only commit when all expected replacements match.
For Flows workflows, we use finer-grained conflict checks. The user may be adjusting one shot node while the Agent modifies another. If any workflow change causes rejection, frequent auto-saves would make parallel editing very hard. Therefore, the system mainly checks whether the node the Agent intends to modify has changed: if the same node has been modified by the user, the Agent must re-read; if changes occurred on other nodes, the Agent is allowed to apply non-conflicting modifications on the latest workflow, with a hint about related changes.
This is not auto-merging different modifications to the same content by both parties, but through appropriate version granularity, letting non-conflicting edits proceed as independently as possible while preventing stale state from overwriting new content.
Two mechanisms work together here: change notifications let the Agent perceive artifact updates in time, and version checks provide final protection at submission. Even if a notification is delayed, the Agent cannot use stale state to overwrite user modifications.
Of course, optimistic concurrency control solves state consistency, not creative-intent conflicts. For example, the user wants a shot to have cool tones, but the Agent continues in warm tones per its original plan. Even without version conflict, which approach ultimately wins still needs to combine the latest user intent and task constraints.
9.4 From Artifacts and Canvas to a Shared Creation Space
In interaction design, we referenced products like Claude Artifacts and ChatGPT Canvas.
These products let generated content stand apart from ordinary chat messages, so users can view and edit documents or code beside the conversation, reducing the cost of repeatedly describing modifications in natural language.
This interaction style is highly relevant to Lomo Agent. Creators often prefer to drag a node, modify a storyboard passage, or replace an image directly rather than re-describing every operation to the Agent.
However, Lomo Agent's shared workspace must handle scenarios beyond ordinary document editing.
On one hand, creative objects are more complex — beyond text, there are images, videos, 3D assets, and Flows workflows with node dependencies. On the other hand, the same work can be referenced by multiple tasks, with simultaneous async generation tasks and direct user modifications. The system must maintain not just content, but versions, associated tasks, and the execution-state awareness of artifact status.
Therefore, we connect the editable workspace with the Agent's execution state, letting users operate on works directly and letting Workers re-fetch valid object states in later execution.
From current practice, an important change brought by the shared workspace is that users can participate in the same creative process through both natural language and direct editing. But this also places more responsibility on the system for state synchronization and concurrency control. Especially when one asset affects multiple tasks, how to manage object dependencies and which downstream artifacts must be regenerated after an upstream change remains an area we continue to explore.
10. Creative Capability Orchestration: From Model Calls to Complete Deliverables
For a creation platform like Lomote AI, the Agent faces not just how to choose and call tools, but also how to understand the relationships among different creative capabilities.
Generating a single image is relatively simple, but producing a full short film may involve script, storyboard, character design, scene reference, shot generation, and post-production. Each stage may use different models, and the result of one stage directly affects subsequent production. More importantly, a single generation task succeeding doesn't mean the whole creative flow will complete.
For example, the Agent plans a production scheme: first generate characters and backgrounds, then cut out the person, and finally composite into a specified video frame. If the platform lacks the corresponding cutout or video-compositing capability, even if earlier material generation goes well, the whole scheme still can't produce a final work.
Therefore, we need the Agent to understand the platform's actual capabilities and the dependency relationships among different tools and artifacts during the planning stage.
10.1 Let the Agent Understand Creative Flows and Capability Boundaries
Lomote AI has integrated multiple image, video, and 3D generation and enhancement models, and provides asset management and Flows visual workflows.
These tools each have applicable scenarios; different models differ in input conditions, generation quality, cost, and output format.
For the Agent, tool selection must consider the current creative goal, existing material, and subsequent production requirements.
For example, when making a short film with a fixed character, the Agent needs to consider how to establish character references, which models support reference-image input, how to maintain visual consistency across shots, and whether results can enter downstream production. If it just calls models directly based on the user's final goal, it can easily produce large amounts of intermediate material that looks good but can't be used in the final work.
To this end, we organize creative experience and tool-capability constraints into on-demand-read guides, covering character consistency, storyboard planning, reference use, workflow orchestration, and the platform's currently supported production capabilities. These guides are provided to the Agent in relevant tasks to help it plan executable creative steps, without keeping all domain knowledge in the system prompt long-term.
This part also works with the Context Engineering introduced earlier: the Agent obtains needed tools and creative knowledge according to the current production stage, then completes planning and execution with actual capabilities.
Of course, relying on the model to understand tool descriptions still has limits. The model may propose seemingly reasonable but actually unexecutable production schemes. Therefore, for key tool capabilities, input/output constraints, and execution conditions, the system still needs to provide explicit validation baselines.
10.2 Multi-agent Video Production from ViMax
In exploring creative capability orchestration, I also looked at ViMax, a multi-agent system for video generation.
ViMax organizes script, characters, storyboard, shot generation, and video assembly into a production pipeline, coordinating related tasks through different roles or execution stages.
This approach is valuable. Video production is naturally staged; different stages need attention to different issues. Script planning focuses on narrative structure, character design on visual consistency, and shot generation on specific model input conditions and generation effects.
Specialized execution division can help organize complex production flows.
However, such architectures also have trade-offs. On one hand, multi-agent collaboration increases context passing, state synchronization, and coordination costs; on the other, a predefined production pipeline may not cover users' ever-changing creative needs.
Lomo Agent faces a more open scenario. Users may only need a poster, may want to generate video from existing material, or may directly ask the Agent to modify a Flows workflow. If we introduce a fixed multi-agent pipeline for all creative needs, it may add unnecessary execution layers.
Therefore, at this stage we lean toward a unified task-execution system, combined with on-demand creative guides and tool capabilities, letting the Worker organize production steps based on the specific goal. For complex creative tasks that need long-term maintenance, we'll gradually explore more explicit specialization and task-dependency management. The trade-off here aligns with the Main/Task design: only when an independent execution space brings real benefit should we add extra coordination layers.
10.3 Production Dependency Graph: From Linear Execution to Dependency Management
Currently, the structured execution checklist helps the Agent maintain creative progress, but when tasks involve many interrelated assets and generation steps, a linear checklist structure still has limits.
For example, a short film has multiple shots that share the same character design but use different scenes and generation models.
Some shots can be generated independently; some must wait for upstream reference material. If the user changes a character's look mid-project, related shots may need regeneration, but unrelated scenes shouldn't be affected.
Such relationships are better expressed as a Production DAG (Directed Acyclic Graph).
In such a structure, each node represents a production step or intermediate artifact, and connections between nodes describe dependencies.
Structuring these relationships further gives the system a chance to realize several capabilities:
- Dependency scheduling: Only execute downstream steps when necessary upstream assets are ready.
- Parallel generation: Shots without direct dependencies can execute in parallel when resources and budget allow.
- Change propagation: When character or scene settings change, locate potentially affected downstream results to avoid indiscriminate regeneration.
- Incremental production: Reuse still-valid materials and generation results as much as possible, reducing repeated calls and production costs.
However, dependency graphs bring additional engineering complexity. Especially since dependencies in creative tasks are not always deterministic: when an upstream asset changes, whether downstream results must be regenerated still depends on the modification scope and specific production requirements.
Also, even if dependencies are correct, results aren't guaranteed to meet visual expectations. Character consistency, shot coherence, and overall rhythm still need independent quality-evaluation mechanisms.
Therefore, Production DAG is more of an evolutionary direction we're exploring. At this stage, we prioritize ensuring the Agent can complete executable creative flows within Lomote AI's existing model and tool capabilities, while controlling the cost and validity of intermediate artifacts.
As task dependencies, automatic acceptance, and multimodal understanding improve, there may be opportunities to combine these mechanisms, moving the Agent from step-by-step task execution toward managing a complete production flow. But judging whether these designs are truly effective requires more than a smooth demo. Complex creative tasks will still involve tool failures, user modifications, state recovery, and duplicate generation.
Whether these designs are effective still needs validation through fault recovery and end-to-end collaboration drills.
11. Engineering Validation: From Fault Recovery to End-to-End Collaboration
Agent systems have a notable trait: a single module working correctly doesn't mean the whole collaboration process is reliable.
For example, a message may route correctly and a tool call return successfully, but the Agent may forget to continue after generation completes; or each step may meet expectations, yet the final result includes multiple duplicate documents, leaving the user unsure which is the latest version.
Such problems are hard to discover through traditional unit tests or a few simple dialogue rounds.
Therefore, during Lomo Agent development, besides validating the correctness of each module, we also tried to let the Agent complete full flows in near-real usage scenarios, observing the actual performance of combined mechanisms.
11.1 Runtime Reliability Validation
For mechanisms like persistent execution, message inbox, async generation, and user authorization, many problems appear only during specific state transitions.
For example:
- A video generation request has been submitted, but the Worker restarts before saving the result
- An old Worker becomes unreachable and is taken over by a new process, then resumes execution
- The user clicks stop exactly when a generation-completion event arrives
These situations rarely appear in normal demos, but can directly affect execution state, user artifacts, and even actual costs. Therefore, during development we used fault-injection tests to focus on validating the system's handling of these abnormal cases.
Specific tests covered the following key execution boundaries:
| Validation Scenario | Key Focus |
|---|---|
| Execution interruption and recovery | Can it recover from already-persisted state and avoid duplicate execution |
| Duplicate delivery and network timeout | Correct handling of idempotent requests and unknown-result situations |
| Worker concurrent takeover | Can old executors be blocked from writing stale state |
| Cancellation and async event races | Can already-stopped tasks be reawakened by late events |
| Budget and authorization changes | Does recovery re-check current permissions |
| Shared-object concurrent modification | Can old versions overwrite user's new modifications |
| Memory correction and revocation | Does invalid info still leak through summaries or profiles |
For model outputs that can be stably reproduced, we used fixed results for replay, focusing on runtime state transitions. External generation services could simulate timeouts, duplicate callbacks, and out-of-order events, avoiding dependence on real paid models each time. The focus here is determinism of system behavior. Even if the model is somewhat random, task state, execution permissions, and billing constraints should have clear and verifiable rules.
A representative test simulated a Worker suddenly "hanging" during tool execution. In the test, the model used a fixed output, first decided to call a query tool; the first Worker saved the model decision and marked the tool call as started, then was artificially suspended. When its lease expired, a second Worker took over, the execution generation rose to 2, and it continued from the checkpoint: since the model decision for that step was already saved, it didn't re-infer but used the same idempotency key to complete the tool call. Finally, the first Worker was restored and tried to submit its late result.
The test result matched expectations: the turn eventually entered completed, execution generation was 2; the model was called only once, only one tool result existed, and the old Worker's state write was rejected. We also covered integration tests for re-delivery after queue message loss, idempotent recovery of post-approval generation tasks, blocking new matter creation from cancelled turns, fee authorization limits, and memory revocation.
11.2 End-to-End Collaboration Drills
Runtime tests validate whether the system handles abnormal states correctly, but cannot fully reflect the actual collaborative experience.
For example, the Agent successfully completes storyboard planning but stops at the next stage until the user types "continue" to start video generation. At the program execution level, this isn't a fault, but for the user the whole process is clearly not natural.
So I introduced scenario drills.
In an isolated environment, using a real text model, simulated generation services, and simulated users with preset behavior, the Agent completes a relatively full creative flow.
Drill scenarios cover series posters, short film production, Flows workflow modification, 3D creation, multi-task switching, historical task recovery, and memory correction/forgetting. These scenarios involve continuous model and tool calls, user confirmation, async waits, and cross-task context association.
For example, in a short-film production scenario, the simulated user first proposes a creative goal, and the Agent must organize storyboards, prepare material, and submit generation requests. During execution, the user may adjust plans, choose shots, or temporarily insert another task. We want to observe not just whether a single step succeeds, but whether the Agent can continue toward the original goal amid these changes.
I focus on:
- Whether the Agent pauses or continues at the right nodes
- Whether users need to repeatedly send "continue" to advance a task
- Whether unnecessary confirmations or duplicate generations occur
- Whether new messages correctly associate with existing tasks
- Whether the Agent gets the latest state after the user modifies a work
- Whether the same work is accidentally split into multiple copies
- Whether old info still influences subsequent collaboration after memory correction or revocation
Compared to calling real media generation services directly, the simulated environment controls test costs and makes it easier to repeatedly construct specific scenarios and observe cross-module interaction issues.
However, this approach has clear limitations. Simulated generation results can't validate real visual quality, and preset simulated users can't fully reflect complex real-life creative behavior. These drills mainly check the continuity of task organization, execution state, and collaboration flow, not directly representing real user task success rates. Final work quality and actual user experience still need evaluation with real generation results and user feedback.
11.3 How to Measure the Quality of a Collaboration
In these drills, I gradually realized that relying only on whether model replies are fluent and tool calls succeed is hard to fully measure a long-running Agent.
Users care more about other questions:
- How many times do I have to repeat myself?
- Did the Agent remember already-confirmed requirements?
- Can it naturally pick up previous work days later?
- Does it know what happened when I modify a work?
So beyond traditional execution correctness, I also began to focus on task continuity, user intervention cost, duplicate operations, and artifact consistency.
Actual drills exposed issues that are hard to find when testing modules individually.
For example, during a series of English posters, Main correctly understood user needs and created a task, but the Worker switched to Chinese in subsequent planning. The problem wasn't task routing, but that the task only received the mainline's processed goal description, missing the language info in the user's original expression. We later adjusted how user language is passed, letting the task maintain reply language based on actual user input rather than relying on the model to infer from the task description.
Another problem appeared at the junction of human checkpoints and async generation. The Agent had advanced to a stage needing user confirmation, but didn't provide an actionable confirmation card; or after the user approved generation, the external task completed normally but the task didn't continue. For these cases, we added a fallback confirmation card in the runtime and auto-wake after generation completes, reducing reliance on the model's interaction behavior.
Individually, these issues may not manifest as obvious execution faults, but they directly affect the collaborative continuity users feel. They also made me clearer: the model is responsible for understanding intent and planning steps, while waiting, authorization, and execution recovery are key states that the runtime must guarantee deterministically.
Of course, these dimensions alone don't represent the Agent's intelligence level. For a creation platform, process completion and work quality must be assessed separately. The Agent can generate all shots as planned, but that doesn't guarantee the final video meets expectations on visual style, character consistency, and narrative rhythm. More reliable multimodal acceptance remains a capability we hope to improve.
From current practice, end-to-end drills provide a relatively controllable validation method. They not only help uncover problems in cross-module collaboration, but also let architectural design principles truly withstand full execution-flow testing. Compared with simply increasing model capabilities, how to make those capabilities connect stably in long-term collaboration is equally worth continued investment.
12. Open Problems and Future Exploration
Lomo Agent currently has basic capabilities like multi-task organization, persistent execution, context management, and shared workspace, but there's still much to explore before reaching my ideal creative agent.
First is cross-task collaboration.
The current Router/Worker structure lets multiple tasks progress independently and access shared artifacts when needed. But if we further allow Workers to actively ask each other for help, delegate tasks, or negotiate shared budgets, we need to solve more complex permission, state synchronization, and concurrency consistency problems.
Multi-agent collaboration looks attractive, but more agents don't naturally mean higher efficiency. The extra coordination overhead and context passing may offset the benefits of specialized division. I want to start from actual needs to find work worth splitting, then gradually expand collaboration scope.
Second is dependency management of creative flows.
The Production DAG mentioned earlier is a direction I'm watching. For complex projects with multiple shots, characters, and scenes, explicit dependency relationships support parallel execution, incremental updates, and result reuse.
However, dependencies in creative tasks aren't always deterministic. When a user modifies a character design, which shots must be regenerated and which remain reusable often require judgment based on actual content.
Async generation reliability also has room for improvement.
Currently, for generation tasks already submitted upstream, the system cannot actively cancel; if a task was created but a fault occurred before saving the upstream task ID, we also lack reverse lookup and recovery association capabilities.
These cases already have state-handling and cost-fallback mechanisms, but may still incur unnecessary generation costs. Going forward, we hope to gradually integrate upstream cancellation and task-query capabilities, letting the system more accurately handle execution interruptions and last-minute user changes.
Multimodal quality acceptance is also hard.
Currently, successful tool returns and checklist-step completion only prove the system performed the operation. True work quality involves visual consistency, motion coherence, audio-video synchronization, and user's subjective aesthetics.
As multimodal models develop, I hope to gradually bring these evaluation capabilities into the execution flow, helping the Agent more accurately decide when to adjust, when regen is worthwhile, and when to hand the decision back to the user.
Additionally, long-term memory accuracy, task routing reliability, and the boundary of proactive follow-up need continuous validation.
Source tracking and version management improve memory controllability but can't fully eliminate model extraction errors or misrecall of historical info. Similarly, while current routing combines task index, history retrieval, and similar-task deduplication to reduce misjudgment, for large accumulations of tasks over time, we still need real usage data to measure task-association accuracy and how often users must correct the system.
For proactive follow-up, the system may know a task has leftover work, but not whether the user wants to be interrupted right now. We hope future Agents can be more proactive, but this initiative must be built on user authorization, task value, and actual usage habits.
These problems may not have mature solutions in the short term. As Lomo Agent enters more real creative scenarios, we'll keep re-examining the current architecture to see which designs are truly effective and which need adjustment.
Conclusion
Looking back at the journey from initial concept to launch of Lomo Agent, I increasingly feel that Agent development is bringing interesting new problems to software architecture.
In the past, when designing software, we typically defined clear functions, flows, and interaction boundaries, then let users follow those rules. When AI becomes capable of understanding intent, calling tools, and making autonomous decisions, much of the organization and coordination work that users previously bore can also be gradually handed to the system.
But improved model capability is only part of it.
An Agent that can collaborate long-term also needs to know what it's working on, what it has done, which information remains valid, and under what circumstances it should continue or stop. It needs to handle interruption, concurrency, cost, and permissions in real systems, and it must allow users to change their minds at any time.
This Lomo Agent architecture design is a practice centered on these problems. Many ideas come from excellent products and engineering practices like DeepSeek, Manus, Claude Code, and Temporal, and some were gradually discovered in my own creative scenarios.
The current implementation is in early exploration; there's still a long way to go from the ideal form, and many designs need to be tested by real users.
However, I always believe that as model capabilities advance, the interaction between software and people still has enormous room for imagination.
Perhaps one day, we won't need to learn how to use a piece of software; we'll just tell it what we want to accomplish.
That is the direction we hope Lomo Agent keeps approaching.
If you're also building Agents, feel free to exchange architecture design and engineering practice experiences;
If you're a creator, welcome to experience Lomo Agent with real creative needs.
Experience Lomo Agent: Lomote AI
References and Further Reading
The links below mainly illustrate architectural inspiration and research background, and do not imply that each project has the same implementation or design goals as Lomo Agent in all capabilities.
- DeepSeek Super-agent Prototype
- Temporal — Design Patterns
- OpenAI — Using Goals in Codex
- Anthropic — Effective Harnesses for Long-Running Agents
- Claude Code — Hooks
- Anthropic — How We Built Our Multi-Agent Research System
- Manus — Context Engineering for AI Agents
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- Zep / Graphiti: A Temporal Knowledge Graph Architecture
- LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
- OpenClaw — Memory Architecture
- ViMax — Multi-Agent Video Generation
Note: "existing mechanisms" and "evolutionary directions" are labeled as far as possible. For external service idempotency guarantees, data covered by forgetting, model caching behavior, and automatic acceptance effects, actual capabilities are constrained by specific implementations and dependencies.
Top 2 of 6 from juejin.cn, machine-translated. The original thread is authoritative.
You write long-term memory as traceable profiles and facts. I haven't run Lomo. When I warm up the poster on Wednesday, does the recalled fact still carry its original item number?
Good question, the answer is no. The source (item and original message) is stored in the memory's source table, for the system to use: cascading undo on forgetting, displaying 'which memory, from which sentence' in the UI. Only the date and content are injected into the model. On Wednesday, 'a bit warmer' can find the original poster because the Router uses the item index to route the message back to the original item, which carries its own context; long-term memory only adds background like 'this person prefers low saturation.' I roughly understand what you mean — the problem you point out exists: the model can't see the source, so a project-specific convention might be taken as a global preference. Currently we only rely on the PROJECT category and time decay to mitigate this risk; later we're considering marking the source item in recall results, or adjusting weights by current item.
Thanks for the explanation.
You wrote about persistent execution; I've hit the other half. The state is saved, but what wakes it up isn't persistent. No cron on mobile, only AlarmManager. When the system reboots, all scheduled alarms are lost. So you must listen for BOOT_COMPLETED and re-schedule every active todo. The one that's missed never rings. Then there's delay. I haven't applied for the exact alarm permission, so I can only fall back to setAndAllowWhileIdle, which can be minutes late under Doze. How late, the system doesn't say. So the tool return must include an exact=true/false flag. Without it, the model's next step will say 'reminded on time.' That lie comes from the permission, not the model. The ledger can answer 'what to do next,' but not 'when will I be woken up to do it.' I haven't gotten around this part.
That's a specific pitfall of the end side. Lomo Agent runs server-side, which is somewhat privileged: the database is the source of execution facts, the queue only handles waking, and if a message is lost a watchdog re-finds it every 10 seconds based on the database state. The watchdog itself re-registers idempotently at worker startup, so 'all alarms lost after reboot' becomes 'woken later' for us, not 'never rings.' Frankly, we've turned 'when will I be woken' into something derivable from the ledger, at the cost of bounded latency. That path doesn't work on mobile, because only the system can wake you, and the system offers no guarantee. On mobile, whether you get woken is decided by the system — I think of three directions for reference. First, separate reminders from execution: pure reminders prepare content at scheduling time and hand them to system alarms or local notifications; tasks that must execute at a specific time go to the server, and the phone just receives results. Second, don't expect a single wake to succeed; instead reconcile every time you wake: boot, update, timezone change, push, opening the app all go through the same idempotent reconcile; anything missed and overdue is immediately caught up, recording how late it was, and all future todos are re-registered; also add a server push fallback, deduplicating by todo id. Third, you can refine exactness: at scheduling time, return what level of guarantee the system gave, so the model says 'around 9 o'clock' from the start; after triggering, return planned time, actual time, and how late it was — the runtime determines punctuality, the model only relays it. That way the ledger still can't answer 'when will I be woken,' but it can answer 'what was missed and how late,' and at least neither the system nor the model has to lie.