Exploratory and Assimilating Reflection: Reflective Recall Cycle for Long-term Memory

2026-07-20

AI Summary

This work addresses the limitations of large language model agents in long-term interactions, where statelessness and constrained context windows hinder effective memory utilization. Existing memory retrieval approaches struggle to efficiently integrate heterogeneous memories and suffer from poor sample efficiency. To overcome these challenges, the paper proposes the Explore-Assimilate Reflection (EAR) framework, which first conducts exploratory reflection to iteratively retrieve initial memories and accumulate experiences, followed by assimilative reflection that replays an experience buffer to optimize a global reranker. EAR uniquely unifies exploration and assimilation mechanisms, achieving both high retrieval performance and strong sample adaptability. Experiments demonstrate that EAR improves memory retrieval accuracy by up to 17.9% over baselines on two long-term dialogue benchmarks, while exhibiting superior sample efficiency and robustness to noise.

Abstract

LLM-based autonomous agents require external memory to overcome their statelessness and limited context window for long-term interaction and dynamic knowledge reasoning. However, existing memory retrieval methods often lack adaptability and sample efficiency, and struggle to retrieve the right mixture of memories from heterogeneous stores. We propose Exploratory-Assimilating Reflection (EAR), a framework for high initial retrieval performance and sample-efficient adaptation. EAR combines two mechanisms: Exploratory Reflection, which performs iterative search to bootstrap retrieval and collect useful experiences for each query, and Assimilating Reflection, which replays these experiences from an Experience Buffer to refine a global reranker more efficiently than methods relying only on immediate rewards. Experiments show that EAR improves retrieval by up to 17.9% over the baseline retriever on two long-term dialogue benchmarks. We also show that EAR is highly sample-efficient and robust to noisy feedback.

Problem

• long-term memory
• memory retrieval
• autonomous agents
• sample efficiency
• heterogeneous memory

Innovation

• Exploratory Reflection
• Assimilating Reflection
• Long-term Memory
• Sample Efficiency
• Memory Retrieval