Open-Source MemT 1.0 Puts Agent Memory In the Forward Pass, Slashing KV Cache 66x
Long-running coding agents can now accumulate procedural experience without paying linear VRAM costs; MemT-1.0's fixed-size native memory keeps storage roughly 66x smaller than a full KV cache at 128K history, which makes continuous multi-round agent work practical on modest GPU setups.
MemT 1.0 replaces the bolt-on approach to LLM memory. Instead of growing context windows or retrieving text from external stores, its MoE architecture embeds a Native Memory Block in selected Transformer layers. The model keeps two memory components: Local Memory, a per-session dynamic hidden state that absorbs the current task's experience, and Hyper Memory, trainable meta-parameters that control read, write, erase, and forget operations. The whole read-write loop is trained over about 34 billion tokens.
The effect on real coding work shows up on SWE-bench Verified. On a Django task, an agent without experience modifies implementations directly and runs into long debugging loops; with memory, it first writes a reproducible test case, locks the bug, then fixes the code and checks edge cases. The all-3-attempts success rate climbs from 54.0% to 62.8%, and the overall pass rate goes from 66.07% to 69.73%. The persistent hidden state stays constant in size, so at 128K history MemT's storage footprint is roughly 66x smaller than a full KV cache. That removes the linear VRAM growth that long-context agents hit. Deployment is standard open-source: clone, create a Python 3.11 env with uv, download the BF16 checkpoint (~70 GB for Nano), start SGLang, and run the memory save/restore example.
The SWE-bench improvement comes with a documented workflow change: the agent shifts from direct code modification to first writing a reproducible test case. That behavioral shift is the strongest evidence that internal memory changes agent strategy, not just raw capability.
The constant-size hidden state is the architectural win: instead of paying linearly for context length, memory cost stays flat. The 66x figure at 128K is impressive, but the bigger point is that the cost curve is decoupled from conversation length.
The closing RSI argument is a strong claim: memory-native models are positioned as a prerequisite for recursive self-improvement. That's a philosophical stance more than a demonstrated result; SWE-bench alone doesn't prove it.
Releasing both Nano and Flash on different base models (Qwen and DeepSeek) suggests the memory mechanism is architecture-agnostic, which could make it a general approach for other MoE models.