LiveMem: Maintaining Memory State Continuity in Long-Running LLM Inference

2026-08-03

AI Summary

Current long-running large language models lack persistent memory during context switches, leading to the loss of historical information. This work proposes LiveMem, which introduces for the first time the concept of “state continuity under context switching.” By constructing a fixed-capacity endogenous memory state decoupled from the active context, LiveMem enables cross-turn information persistence. The approach integrates memory-oriented post-training with a state-aware serving mechanism, jointly leveraging a limited KV cache window and external memory for inference. Experiments on LongMemEval demonstrate that the model can accurately answer questions even when supporting evidence has been evicted from the current context, significantly validating its ability to retain and utilize information beyond the active window.

Abstract

Long-running assistants and agents consume interaction streams that eventually outgrow the context. Existing context retention, summarization, and retrieval preserve access to selected history, but do not provide a persistent state over the full lifecycle when working context changes. We formulate this missing inference capability as \emph{state continuity under context turnover}: carrying computation forward through a fixed-capacity memory state whose lifetime is independent of the active context. We introduce an intrinsic memory method, \textbf{LiveMem}, which augments a pretrained full-attention LLM with a memory state that preserves the historical information over the whole lifecycle while the main attention path retains a bounded KV window. Context turnover and memory state maintaining, memory-oriented post-training, and state-aware serving jointly make this memory state load bearing after its originating tokens are released. Our experiments show that LiveMem achieves leading overall performance among evaluated systems and other intrinsic memory methods. Experiments on LongMemEval show that LiveMem is able to answer the question based on the memory state, even when the supporting evidence has been removed from the current context, and evidence-distance analysis shows that useful information persists beyond the active window. LiveMem thus establishes state continuity as a distinct and complementary abstraction for continual LLM inference.

Problem

• state continuity
• context turnover
• long-running LLM inference
• memory state
• persistent memory

Innovation

• state continuity
• intrinsic memory
• context turnover
• memory-augmented LLM
• long-running inference