Artificial Intelligence

Persistent Memory for AI Agents: User Profiles, RAG Retrieval, and the Two-Layer Architecture That Enables Proactive Service

By 前端的阶梯 · Oct 22, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Agent products that cannot remember users across sessions stall at reactive Q&A; the two-layer architecture here is a concrete path to proactive service, and the PR-based update flow gives production teams a way to keep memory and knowledge trustworthy under multi-tenant constraints.

Summary

Persistent memory for AI agents splits into two scales that share the same machinery. User memory extracts stable facts about one person from conversation and stores them as atomic notes, narrative paragraphs, JSON cards, or executable Python state. Knowledge bases hold shared domain knowledge and run on the full RAG stack: chunking, dense and sparse embeddings, hybrid fusion, and neural reranking. Both scales hit the same failure modes: conflicting facts, stale knowledge, and retrieval misses.

A three-level evaluation framework (basic recall, multi-session retrieval, proactive service) shows why proactive service is hard: it needs both a global overview and precise details at the same time. The architecture that survives the experiments is two-layered. Advanced JSON Cards keep a few critical facts structured and resident in context; contextual retrieval, which prepends LLM-generated background prefixes to chunks, pulls exact dialogue details on demand. Anthropic's numbers appear in the same argument: prefixing chunks cuts retrieval failure by 49% with BM25 and 67% with a reranker.

Knowledge maintenance gets the same discipline as code review. Every update is a PR proposed by one agent and independently reviewed by a second agent from a different model family, with raw evidence as ground truth and derived indexes rebuilt only after merge. Retrieval-side security is explicit too: retrieved content is data, not instructions, and side-effectful actions need a separate authorization decision.

Takeaways

— User memory and a shared knowledge base solve the same problem at two scales: individual vs. collective, with the same underlying vector retrieval and knowledge compression.

— LoCoMo, a benchmark for long-term conversational memory, uses dialogues averaging about 300 turns across up to 35 sessions and tests QA, event summarization, and multimodal dialogue generation.

— The proposed three-level memory evaluation framework runs from basic recall to multi-session retrieval to proactive service; each level in the companion test set has 20 cases.

— Four storage formats form a progression: Simple Notes, Enhanced Notes, JSON Cards, and Advanced JSON Cards; Advanced JSON Cards wins on disambiguation but costs more to generate and maintain.

— User as Code represents memory as typed Python state regenerated from an append-only fact log, turning aggregation, conflict detection, and constraint checks into deterministic code.

— Mem0 v3 switched from write-time conflict resolution to append-only writes plus hybrid retrieval at query time, raising LoCoMo from 71.4 to 92.5 and LongMemEval from 67.8 to 94.4.

— Memobase separates a developer-configurable user profile from an event timeline and batches memory extraction to amortize LLM calls.

— BM25 improves TF-IDF with term-frequency saturation and length normalization; k1 controls saturation and b controls normalization strength.

— Hybrid retrieval fuses dense and sparse results with normalized weighted scores or reciprocal rank fusion (k=60), then applies a cross-encoder reranker for final ranking.

— Contextual retrieval prepends LLM-generated context to each chunk before indexing; Anthropic reports 49% lower retrieval failure with BM25 and 67% with a reranker, at roughly $1 per million document tokens with prompt caching.

— OpenViking organizes knowledge as a virtual filesystem with L0/L1/L2 summaries and Git-versioned Markdown; it only works if entries cross-link and index pages are maintained.

— The two-layer memory architecture that reaches proactive service keeps key facts in Advanced JSON Cards resident in context and uses contextual retrieval to fetch raw conversation details on demand.

Conclusions

Moving conflict resolution from write time to query time, as Mem0 v3 does, trades a tidy memory store for auditability: old facts survive as timestamped entries and the retriever has to decide which one is current. For user memory, that is the safer default because a wrong DELETE is unrecoverable.

Treating knowledge updates as a two-agent code review is a process answer to a model problem: LLM-generated edits can silently destroy information, so the fix is evidence-backed diffs, cross-model reviewers, and CI before merge rather than a smarter prompt.

The black-cat/white-cat and Xfinity examples make the general point that raw chunks are not knowledge. Similarity retrieval cannot count, generalize, or exclude, so aggregation and rule extraction must happen at index time.

Storing multimodal embeddings in context at one token per face or voiceprint is a surprisingly cheap way to give agents perceptual memory; the tradeoff is that it only works within the context window, while Engram-style parameter insertion scales further but needs special pretrained models.

The two-layer memory architecture resolves the structured-versus-unstructured debate by refusing to choose: always-on JSON Cards supply the overview, contextual retrieval supplies the detail, and neither alone satisfies the proactive-service level defined earlier.

Concepts & terms

Retrieval-Augmented Generation (RAG)

A pattern that retrieves relevant fragments from an external knowledge base and injects them into the LLM's context so answers can use up-to-date or private information beyond training data.

Append-only log

A data structure where new entries are only added at the end; existing entries are never modified or deleted. Used for trajectories and Mem0 v3 memory to preserve history.

LoCoMo

A benchmark for long-term conversational memory built from very long multi-turn dialogues (about 300 turns, up to 35 sessions), testing QA, event summarization, and multimodal dialogue generation.

Cosine similarity

A measure of semantic closeness between two embedding vectors, computed from the angle between them; values near 1 mean similar direction and meaning.

BM25

A sparse retrieval ranking function that scores documents by exact keyword matches, with term-frequency saturation and length normalization; a common production default in keyword search.

Reciprocal Rank Fusion (RRF)

A fusion method for hybrid retrieval that combines result lists by summing 1/(k + rank) for each document, ignoring raw scores; k is a smoothing constant, often 60.

Cross-encoder

A reranking architecture that concatenates query and document into one input and lets the model compare them token by token; slower but more accurate than bi-encoder retrieval.

GraphRAG

A retrieval-augmented generation approach that models documents as a knowledge graph of entities and relationships, enabling multi-hop reasoning and entity disambiguation.

Contextual retrieval

Anthropic's technique of prepending an LLM-generated context prefix to each chunk before embedding and indexing, reducing retrieval failures from lost context.

Agentic RAG

An approach where an agent controls retrieval iteratively through a think-act-observe loop, decomposing complex questions and refining searches instead of running a single retrieve-then-generate pass.

Indirect prompt injection

An attack where malicious instructions are hidden in a document or web page that gets retrieved into context, tricking the model into following them.

From the discussion

Featured comments

创了么OPC

Single-session context and cross-session memory are indeed two different things. Making user memory into "what kind of person this is" rather than storing every sentence is a clearer analogy than piling up chat logs. On the knowledge-base side, it is shared domain material for everyone; conflicts, staleness, and biased retrieval will trip up both sides together. I would especially like to see how, after memory compression, old preferences and newly stated preferences clash — which one wins. If the article had eviction rules, it would feel more grounded.

前端的阶梯

Compression solves "what to store, what to delete"; conflict resolution solves "when two entries don't match, which one to trust" — the article deliberately did not write it as an unconditional "always trust the latest." Current consolidation tends to keep only the latest; Mem0 v2 does UPDATE on write, while v3 keeps both entries and decides the current one at query time by recency/retrieval evidence; the knowledge-base side emphasizes scenario constraints, so both can be valid. Eviction from compression mainly relies on access, decay, and uniqueness; an old summary should not override a preference the user just corrected. What you care about — concrete implementation (type × conflict × deletability) — is indeed the next cut to add, agreed.

创了么OPC → 前端的阶梯

Type, conflict, and whether it can be deleted — these three columns are more useful than piling up another set of storage terms. In my implementation, I would mark a preference the user just corrected as something that cannot be overwritten by an old summary, and pick the current one by time at query time, so compression does not swallow back the tone they just changed.

See top comments, translated →

Source: juejin.cn ↗ Google Translate ↗ Backup ↗