Open-Source MemT 1.0 Puts Agent Memory In the Forward Pass, Slashing KV Cache 66x
Using LLMs for a long time, you've surely hit this problem: the thing you just asked the big model to do, switch conversations, and it's lost. Your preferences, project background, past pitfalls, you have to explain it all over again.
At the beginning of this year, I realized this problem, so I said the core problem I want to solve this year is the model's memory capability.
Later, people started expanding context windows to 1M, stuffing more history in; then RAG and various external memory systems became common, keeping history and retrieving relevant text into context when needed. While these solutions solve many problems, the evolution of Agents pushes the memory problem of LLMs to the forefront: we want the model to remember not just what was said in the past, but what it did.
The Agent field already has no shortage of bolt-on memory solutions, but the recently released open-source model MemT 1.0 is most interesting because it puts memory read and write directly into the model's computation process.
This raises a core question that runs through everything: When memory is no longer just an appended chunk of retrieved text, but directly participates in the model's own computation, what changes for Agents?
Meet MemT
MemT is a memory-native foundation model. It just released version 1.0. Although just released, total downloads hit 100K in 8 days, which is very hot. So I looked into MemT to talk about it.
It uses a MoE architecture and deeply integrates memory writing, maintaining, updating, and reading into the model's forward computation. That is, the model retains key experiences during practical interaction, converts them into internal numerical states written into session memory, and in subsequent steps they directly participate in computation and invocation.
Behind MemT-1.0 is the Memory Tensor team that built MemOS. Their earlier MemOS is a memory system for LLMs and Agents, involving memory storage, updating, retrieval, and cross-task experience reuse.
In July this year, they also published Metis, the world's first memory-native model, which adds persistent, dynamically updatable memory states into the base, letting the model store and use history through its own computation.
MemT-1.0 advances along the Metis path.
Before diving into the specific model architecture, let's look at the most direct programming experiment: what happens to an Agent's action pattern when relevant experience is written into native memory?
MemT-1.0 proposes an Agent framework specifically for reusing procedural experience in long-horizon programming tasks:
Using the representative open-source large project Django from software engineering benchmarks:
- Without experience guidance: the Agent often behaves like a disorganized novice, searching the codebase to locate the target function and then eagerly modifying the implementation logic. This approach takes a long debugging path and easily introduces bugs; often after multiple rounds of errors it hastily adds verification, with very high trial-and-error cost.
- After writing relevant experience into memory: the Agent's execution steps change qualitatively. Inspired by past debugging logic, it acts like a senior engineer, forming a clear action loop: first actively writing a test case that reliably reproduces the problem, locking the bug and acceptance benchmark, then precisely modifying the core business code, and finally returning to verify edge cases.
Reflected in data, on SWE-bench Verified, a benchmark for real code repair ability, the key metric for Agent consistency and reliability, the proportion of all three attempts succeeding (All 3 Trials / pass^3), jumps from 54.0% without experience to 62.8% (overall average success rate also rises from 66.07% to 69.73%).
Next let's look at the underlying architecture: how does MemT achieve all this through internal model computation?
Three Memory Routes Compared
Before going deeper, let's compare the three mainstream memory approaches:
| Route | Where historical info lives | How it's used when answering |
|---|---|---|
| Long context | Current input and corresponding history cache | Accesses full history token by token via attention |
| RAG external memory | External storage like text, vector DBs, knowledge graphs | Retrieves relevant snippets and stuffs them into the current context prompt |
| MemT native memory | Continually updated numerical hidden states inside the model | Directly read and fused during the model's forward pass |
These three routes are not completely separate. MemT can also combine them in practice: for example, a programming Agent can first retrieve relevant procedural knowledge from an external experience library, then use model computation to inject that experience into its own native memory, making it an internalized state for subsequent reasoning.
Architecture Analysis
In MemT, the memory mechanism revolves around two core concepts:
- Local Memory: each session's own dynamic hidden state, continuously updated during interaction, storing the current task's specific experience and context state.
- Hyper Memory: meta-parameters pre-trained on massive data, providing the control mechanism for read, write, erase, and forget, teaching the model how to remember and how to use.
MemT currently offers two open-source versions based on different base models:
| Model | Base | Total params / Active params |
|---|---|---|
| Nano | Qwen3.6-35B-A3B | 36B / 3B |
| Flash | DeepSeek-V4-Flash | 288B / 17B |
The following diagram shows MemT's full core architecture:
We can string the whole computation into a read-write loop:
- Parallel structure (left and center of diagram): MemT embeds a
Native Memory Blockin selected Transformer layers. While processing input, the native Self-Attention handles the current context text, while the Memory Block in parallel interacts with historical memory. - Memory read and fusion (lower right): based on current text features it generates a query vector, directly reading relevant historical states from the Local Memory matrix. The read result, after normalization and gated weighting, fuses with the native text attention representation into a unified feature vector that participates in the next answer and decision generation.
- Memory write and revision (upper right and middle right): at the end of an interaction round or when processing each token, the system does not simply mechanically append text to the history list. Instead, it feeds the current interaction representation into a state evolution rule based on
Stable GDN-2. Gating parameters provided by Hyper Memory precisely control how much new feature to write, how much old redundancy to erase, and how much outdated association to forget, dynamically updating Local Memory state.
This read-write mechanism is not a simple rule heuristic. It is trained over a total of about 34 billion tokens across pre-training and post-training phases, ensuring robust memory retention and extraction.
Evaluation
Official benchmark data:
Among evaluated parametric memory methods, MemT-1.0-Flash and MemT-1.0-Nano take first and second place respectively.
Beyond the SWE-bench improvements in task decision quality and stability, MemT also shows significant advantages in storage and inference overhead:
- Traditional long-context approaches rely on growing KV Cache, with VRAM usage linear in history length.
- MemT's dynamic Local Memory combined with trainable Hyper Memory keeps persistent hidden state size constant.
- At 128K history length, compared to storing the full context via KV Cache, MemT reduces storage overhead by about 66x.
This lets Agents handle ultra-long, multi-round diagnostic and execution tasks while retaining key experience and avoiding the twin problems of secondary computation bloat and VRAM explosion.
Local Installation and Deployment
How does one deploy MemT? The official inference guide is clear. Here's a single-node deployment example of the Nano version under the SGLang framework:
Server environment requirements: Linux, NVIDIA CUDA, and Python 3.11 or higher. Recommended starting hardware: 2 GPUs for Nano, 8 GPUs for Flash.
First, in an environment with uv installed, clone the repo, create a virtual environment, and install server dependencies:
git clone https://github.com/MemTensor/MemT-1.0.git
cd MemT-1.0/inference
uv venv --python 3.11
source .venv/bin/activate
uv pip install '.[server]'
Next, download the complete Nano checkpoint. Using the Hugging Face Hub client included with the dependencies, store it in the models folder:
python - <<'PY'
from huggingface_hub import snapshot_download
snapshot_download(
repo_id="MemTensor/MemT-1.0-Nano",
local_dir="./models/MemT-1.0-Nano",
)
PY
The full model directory must contain BF16 weights, config files, and tokenizer files. The Nano repo is about 70 GB, so reserve enough disk space.
Start the service after download:
sh start_memt_nano.sh ./models/MemT-1.0-Nano
Default service port is http://127.0.0.1:30000. Session memory snapshots are saved under ./memt-state on the server.
Keep the background service running. Open a new terminal, enter the same directory, activate the virtual environment, and run the official memory save/restore example script:
source .venv/bin/activate
python examples/commit_history.py ./models/MemT-1.0-Nano
If you have the Linux GPU environment ready, you can also copy this prompt directly to an AI assistant that can operate a terminal:
Follow the official inference docs at https://github.com/MemTensor/MemT-1.0/blob/main/inference/README_zh.md, deploy MemT-1.0-Nano on this Linux GPU server, first check CUDA and VRAM, then create a Python 3.11 environment, install dependencies, download the complete BF16 weights, start SGLang, and run the official memory save/restore example to verify results.
Closing Thoughts
Back to the original question: what does putting memory read/write into the model computation actually mean?
For a long time, the main paths to stronger models were scaling parameters, feeding massive data, and lengthening context windows. With long context and external retrieval, models can remember what users said, or fetch a historical reference when asked.
But when LLMs evolve into Agents that need to act continuously, this changes.
Users care not only about chat memory, but about the procedural experience an Agent accumulates through repeated attempts: how to most effectively fix a similar bug? What steps should problem-solving follow? Which implementations have hit pitfalls?
If every task starts an Agent from scratch like a newborn, blindly modifying and testing, no real experience accumulates. This is why the tech community is now densely discussing RSI (Recursive Self-Improvement): for AI to improve itself, it must first have a way to internalize what it has done into a continuously reusable capability. Otherwise each round is a one-off spinning in place, not genuine recursion or evolution.
MemT explores exactly that: making memory's store, retrieve, and modify settle directly into the model. That might be the most promising future of memory-native foundation models.
For the full open-source code and technical report, see Hugging Face and GitHub:
- Hugging Face: https://huggingface.co/collections/MemTensor/memt-1
- GitHub: https://github.com/MemTensor/MemT-1.0