MemFit: Efficient Long-Term Agentic Memory

2026-09-30

AI Summary

This study addresses the inefficiency and loss of detail in existing long-term memory systems for large language models (LLMs), which rely on costly agent-based write operations. To overcome these limitations, this work proposes an efficient memory architecture that employs append-only storage to preserve raw dialogues, combined with segmented summary indexing to enable instantaneous insertion without LLM invocations. Furthermore, a multi-path hybrid retrieval strategy is designed, integrating lexical-semantic signals with cross-encoder reranking. Experimental results demonstrate that the proposed system achieves state-of-the-art performance across three major benchmarks while significantly reducing both the temporal and computational overhead associated with memory construction. Ultimately, this approach provides a scalable, low-latency solution suitable for large-scale applications.

Abstract

Long-term memory systems for large language models (LLMs) have gained popularity for extending reasoning capabilities across applications. Current memory systems rely on LLM agents to organize and consolidate memory, resulting in costly, inefficient write operations. To address this limitation, we propose MemFit, a long-term memory system for conversational agents that reduces the cost and latency of memory operations. Unlike existing systems that rely on expensive LLM calls for memory construction or discard surface-level details through compression, MemFit stores each turn verbatim in an append-only store with near-instantaneous, LLM-free insertion, indexing turns with segment summaries rather than replacing them. Additionally, MemFit uses an LLM-free, multi-path retrieval strategy that combines lexical and semantic signals with cross-encoder reranking over caption- augmented episodes in both textual and multimodal settings. Empirical results on three widely used benchmarks, LoCoMo, MemGallery, and LongMemEval-S, show that MemFit achieves state-of-the-art performance while reducing memory construction time and cost several-fold, providing a scalable and efficient solution for persistent agentic memory.

Problem

• long-term memory
• large language models
• memory efficiency
• conversational agents
• memory retrieval

Innovation

• Long-Term Memory
• LLM-free Retrieval
• Append-only Store
• Multi-path Retrieval
• Cross-encoder Reranking