Background

Based on Huolala's practical experience, this article systematically introduces a four-layer progressive memory architecture for conversational AI agents, achieving a paradigm shift from "passing state" to "cognitive self-evolution":

• Layered memory: A progressive processing chain of data foundation → perception layer → decision layer → profile layer that gradually refines raw dialogue streams into structured cognition directly consumable by models, transforming "raw materials" into "nutrients".

• Cognitive flywheel: Profile dimensions evolve in real time with thousands of individualized variations; communication strategies automatically eliminate the weak and retain the strong based on an effect feedback loop. The system understands users better and becomes more precise the more it is used, building a data-driven self-growth intelligence loop.

01 Why Build a Memory System

1.1 Background

Huolala users are typically active across multiple touchpoints (App, mini-program, H5, phone, instant messaging, etc.), forming a naturally high-value user pool. Most user behavior revolves around "placing an order", with accompanying behaviors such as receiving coupons, inviting friends, and repeat purchases extending beyond the core ordering action.

User conversion typically follows a long chain spanning days or even weeks (inquiry/guidance → coupon/price comparison → order/activation → repeat purchase/invitation). From initial contact to final conversion, users may experience 3–5 session interruptions across phone and instant messaging channels. The longer the chain and the more breakpoints, the higher the demands on the dialogue system: it must remember which stage the user is in, what was discussed last time, and why they hesitated or refused before; understand the user's true intent and emotion in the moment; and speak with communication strategies that vary by individual. All of this presupposes memory. Without memory, the subsequent "understanding" and "appropriateness" are impossible.

1.2 The Octopus Tragedy

Octopuses are geniuses, but each one is an isolated, independent genius. They can: open bottle caps, mimic 15 kinds of creatures, recognize human faces, remember their caretakers, and some experiments suggest they even "dream".

But octopuses have an unavoidable shortcoming—they cannot pass down "wisdom": a mother octopus that hunts does not teach its young to hunt; an octopus that uses a coconut shell as armor does not pass that trick on. All "experience" must be accumulated from scratch by each individual across its lifetime.

This is exactly the tragedy common dialogue systems are repeating:

• Each conversation is like an "isolated octopus"—no matter how smooth this turn goes, the next turn resets everything;
• A user who rejected "too expensive" yesterday enters through a different channel today, and the system recommends from scratch again;
• User A is moved by "revenue cases", user B is immune to "cheapness"—such experiences are scattered across tens of thousands of conversations, with nobody aggregating or inheriting them.

An AI without memory is like an octopus without intergenerational transmission: no matter how intelligent the individual, the species always starts from zero.

1.3 A Real "Incident" Review

We sampled 50 conversations deleted by users in the instant messaging channel and reviewed them. The reasons users deleted them were not at the surface level of strategy, prompts, or knowledge base, but at a deeper level: the AI had no memory, so it could not make cross-channel or cross-session judgments. It could not remember the user's last sentence or the reason for the previous rejection, and could only mechanically "see state A → jump to strategy B", ultimately pushing the user away.

02 How to Build a Memory System

2.1 How a Gold Consultant Remembers a User

During an early research phase, a gold consultant said:

"I don't need the customer to say 'I've refused three times'. From tone, response intervals, and excuses, I know whether they are genuinely busy or think it's too expensive, whether they truly don't need it or just need one more push."

It turns out gold consultants never make decisions based on "tags" alone; they rely on multi-modal, multi-level, cross-time memory reasoning to understand users. Breaking down his statement, a complete 4-layer cognitive chain emerges:

Stage	Focus
Listen to raw tone	Voice signals, unprocessed facts
Observe at the time	Intent/emotion judgment of a single message
Judge in the moment	Synthesize context to decide whether to push or hold
Long-term cognition	Who the user is, what preferences they have

This gave us two key insights:

a. Memory is continuous: what we need to record is not the outcome "user rejected", but the complete process of change from "evasive" to "loosening up".

b. Memory is layered: information has priority and should be stored progressively in four layers: "raw data / immediate observation / current judgment / long-term profile".

2.2 Common Industry Practices

2.2.1 Common Memory Systems

Most dialogue systems use a "state passing" model: abstract key events in the conversation into flat tags (e.g., "user state = not ordered", "rejection reason = busy", "intention = high") and inject them into the LLM as prompt variables. This works for simple tasks but has fatal problems for complex ones:

Problem Dimension	Manifestation
Flat data	Only stores conclusion tags, not process information (tone, rejection frequency); the model sees isolated tags rather than a complete evidence chain.
Emotion not traceable	Only looks at the literal current message, cannot correct based on history (e.g., "OK, OK" is actually perfunctory); misjudgment and forced pursuit lead to user churn.
Cross-session discontinuation	Only records "last rejection", not the interrupted stage and context; recovery is equivalent to starting over.
Cross-channel not unified	Phone and instant messaging each store their own memory; the same user is treated as two new people across channels.
Coarse personalization granularity	Does not distinguish "genuinely busy vs. making excuses" or "price-sensitive vs. trouble-averse"; one size fits all.
No self-evolution	Strategy effectiveness is a black box; optimization relies entirely on manual review, with iteration cycles of 2–3 weeks.
Token inefficiency	Irrelevant fields are fully injected into the prompt, while key information is missing; and the entire history is resent every time.

2.2.2 What Makes a Good Memory System

A good memory system does not stuff everything into the context; it keeps only the most relevant information and stores the rest for on-demand access, making the model more focused, efficient, and intelligent.

For a specific business scenario, this is also a "newborn octopus": every time a user opens a new conversation, it's a "reincarnation" for the AI—if it doesn't remember what the user said last week, doesn't know why the user refused last time, and doesn't know the user's preferences, emotions, and rhythm, everything starts from zero. This time, we want to give the system the ability to "break the octopus tragedy"—to remember, accumulate, and evolve.

To this end, we redesigned a 4-layer progressive memory architecture based on our research—the core idea is:

Upgrade from "flat tags" to a progressive processing chain of "raw data → immediate observation → current judgment → long-term profile", where each layer builds on top of the upper layer.

2.3 Architecture Overview

Layer	Name	Responsibility
L0	Data foundation	Store only, do not judge: raw messages, dialogue transcripts, event streams (100% scene reconstruction).
L1	Perception layer	Observe only, do not decide: intent, emotion, trace-back signals, behavior completion of each message.
L2	Decision layer	The sole dispatcher: stage state machine, strategy library, rule library; it decides "which path to take".
L3	Profile layer	Long-term memory: 6 core user dimensions (price sensitivity, trust level, operational proficiency, preferred time slot, busy tag, response style).

Progressiveness is reflected in two dimensions:

L0 real-time persistence → L1 near-real-time analysis (millisecond-level after each message) → L2 real-time dispatch (after each message) → L3 24h scheduled + real-time fine-tuning

From "raw data" to "immediate observation (intent understanding)" to "current judgment" to "long-term cognition", each layer builds on top of the upper layer.

2.4 Memory Layer Definitions

2.4.1 L0 Data Foundation — Record Only, No Processing

L0 is the factual layer of the entire memory system, faithfully recording raw dialogue and event information without any processing or judgment:

• Raw messages: every message from user/AI with text, timestamp, message ID
• Dialogue transcripts: complete conversation flow, usable for trace-back

2.4.2 L1 Perception Layer — Observe Only, Do Not Decide

Intent (coarse-grained intent + fine-grained intent)

2.4.3 L2 Decision Layer — The Sole Dispatcher

2.4.4 L3 Profile Layer — Offline Consolidation

24h scheduled: full recalculation of all 6 dimensions at 2:00 AM daily (overwrite UPDATE)

In-session real-time: when a user sends ≥5 messages, immediately recalculate the two high-frequency dimensions: busy tag and response style

2.5 Cross-Channel Memory Integration

The same user appears across App, mini-program, H5, phone, and instant messaging channels, but in implementation they are easily treated as multiple independent conversations with completely isolated memory. We made horizontal integration:

a. Identity layer integration: Use the user identity identifier as the primary key to associate all phone and instant messaging sessions.

b. Profile layer integration: L3 profile is channel-independent—dimensions like price sensitivity and trust level are shared across channels; during the nightly recalculation, data from both channels is merged.

c. Stage layer bridging: At the end of a phone call, user state is written to the L0 table; at the start of an instant messaging session, the last user state is read to decide which guidance target to enter from (order/coupon/activation/share).

2.6 How the Memory Mechanism Is Used

2.6.1 How to Generate Strategy Decisions

Take the example of user Mr. Li sending "Why do I still have to pay?":

2.6.2 How Strategies Self-Evolve

03 How Well Did the Memory System Perform

3.1 Metrics

Dimension	Metric	Calculation
Conversion	Task conversion rate	Converted users / reached users
Dialogue efficiency	Average message count	AVG(total messages) WHERE output='task conversion'
AI response latency	Average response time (message arrival → AI reply sent)
User experience	Churn rate	Churned / new total
Personalization	Profile coverage	Users with updated profiles / active users
Strategy self-evolution	Strategy elimination rate	Strategies retired in period / total strategies
Strategy progression rate	Strategies that advance to next stage (within 24h) / uses of that strategy

3.2 Experimental Results

Summary: The progressive architecture costs "0.2s extra per turn" but delivers "conversion +1.7pp, churn -1.6pp, profile coverage from 0 to 75%"—worth rolling out fully.

Dimension	Metric	Common memory strategy	Progressive memory architecture
Experiment info	Duration	Experiment duration: 2 weeks
Method	Account split
Conversion	Task conversion rate	58.3%	60%
Dialogue efficiency	Average messages	18.4	12.1
AI response latency	0.25s	0.45s
User experience	Churn rate	2.8%	1.2%
Personalization	Profile coverage	— (no profile capability)	75%
Strategy self-evolution	Strategy elimination rate	— (relies mainly on manual review)	1% (two strategies retired)
Strategy progression rate	72%	80%

04 Summary

1. Cognitive transformation: technology always evolves. Our biggest gain this time is not just the improvement in key metrics, but a shift in mindset. Every octopus is a genius, but each must accumulate experience from scratch—this is the price they pay in evolution. For AI, we don't have to follow that path. AI can stand on the shoulders of previous generations, understanding people better with each generation.

2. Building memory: from "passing tags" to "building memory"—AI's next breakthrough lies not in more accurate recognition but in deeper understanding. Upgrading from L1 single-message intent recognition to L0-L3 cross-temporal reasoning gives "context" and "time" back to the AI. It can then move from mechanical execution to being a warm communicator, and strategies automatically eliminate the weak and retain the strong based on effects.

3. Capability upgrade: memory system application is never a single-scenario build. This "layered memory + cognitive flywheel" foundation can evolve and be reused across the entire user lifecycle (from acquisition, activation to retention and churn recovery), across phone and instant messaging channels, and in any agent scenario that requires "long-term understanding of people and continuous dialogue"—transforming AI from a single-scenario tool into a cognitive infrastructure that understands people better the more it is used.

Authors: AI Application Team | Zheng Qixin, Wu Hulong, Zhang Guangfa