construct hierarchical memory

Transformers exhibit inherent limitations in modeling long-range context, continual learning, and knowledge integration. To address these challenges, we propose a neuroscience-inspired memory-augmented unified framework that integrates multi-timescale memory, selective attention, and synaptic consolidation mechanisms—shifting from static caching to adaptive online learning. Methodologically, we design a synergistic architecture combining attention fusion, gating control, and associative retrieval, supported by a hybrid memory representation comprising parametric encoding, state-based internal representations, and explicit external memory. We further introduce a hierarchical buffering structure and a surprise-driven memory update strategy to mitigate capacity bottlenecks and catastrophic forgetting. Experiments demonstrate substantial improvements in long-sequence modeling stability and cross-task knowledge transfer. Our framework provides a scalable, biologically plausible pathway toward intelligent models capable of lifelong learning.

Current research on memory mechanisms in large language models remains highly fragmented and lacks a unified theoretical framework. This work proposes an architecture-centered taxonomy that systematically models memory along three orthogonal dimensions: representation, update dynamics, and persistence, formally characterizing core processes such as writing, routing, state transition, and integration. By constructing the first three-dimensional unified framework that integrates implicit/explicit, offline/online, and short-term/long-term memory, it clarifies the boundary between computationally coupled memory and independently addressable memory. Through a systematic literature review, architectural analysis, and multidimensional evaluation, the study synthesizes techniques including attention mechanisms, recurrent states, parameter-efficient fine-tuning, and scalable retrieval-augmented storage, thereby establishing a coherent paradigm for memory modeling and providing a theoretical foundation and design principles for future scalable and adaptive large language models.

Existing benchmarks for long-term memory evaluation primarily focus on factual recall and simple retrieval, failing to adequately assess large language models’ (LLMs’) capacity to organize and leverage complex memory structures. To address this gap, this work introduces StructMemEval, a novel benchmark that systematically evaluates LLM agents’ ability to construct and utilize structured long-term memory in complex reasoning scenarios. StructMemEval emphasizes structural organization through tasks such as transaction ledgers, to-do lists, and tree-structured data. Experimental results demonstrate that mainstream LLMs struggle to autonomously organize memories into coherent structures without explicit guidance, whereas memory-augmented agents equipped with structural prompts achieve significantly higher task success rates. This benchmark thus fills a critical void in the evaluation of sophisticated memory architectures for LLM-based agents.

Existing sequence models struggle to emulate human memory consolidation mechanisms, limiting their ability to efficiently process long contexts during inference while preserving multi-granular memory representations. Inspired by the neuroscientific hypothesis of memory transformation, this work proposes Mela, a novel architecture that employs a Hierarchical Memory Module (HMM) to disentangle high-frequency details from low-frequency abstract information. By integrating cross-frequency coupling and a MemStack feature distribution strategy, Mela enables online, hierarchical memory consolidation without increasing token count. To our knowledge, this is the first approach to incorporate human-like memory consolidation into sequence modeling. Integrated into a Transformer decoder, Mela significantly outperforms baseline models under a fixed 4K pretraining context length and maintains stable performance even when evaluated on substantially longer test sequences.

This work addresses the construction of human-like memory mechanisms in large language models and multimodal foundation models to support continual learning, personalized reasoning, and cross-modal consistency. It proposes the first unified taxonomic framework that systematically integrates three major memory paradigms: implicit memory (e.g., parameterized memory), explicit memory (e.g., external retrieval and graph-structured knowledge bases), and agent memory. The framework is further extended to multimodal settings, elucidating the critical role of memory in cross-modal alignment and agent collaboration. Through a comprehensive literature review and taxonomic analysis, the study surveys existing technical approaches, evaluation benchmarks, and core challenges, thereby establishing a theoretical foundation and offering clear directions for future research on human-like memory systems in artificial intelligence.

Existing studies offer conflicting conclusions regarding the effectiveness of graph-based structures in dialogue memory systems, making it difficult to isolate key design factors. This work proposes the first unified and modular framework for analyzing dialogue memory mechanisms, accommodating both graph-based and non-graph approaches. Through staged controlled experiments on LongMemEval and HaluMem, the study systematically evaluates the impact of core components—memory representation, organization, maintenance, and retrieval. Findings reveal that performance differences primarily stem from underlying system configurations rather than specific architectural innovations. Moreover, the analysis identifies several robust and consistently strong baselines, establishing reproducible benchmarks and actionable design guidelines for future research in dialogue memory systems.

This work addresses the lack of cross-interaction memory continuity in existing long-lived AI agents, which struggle to effectively preserve, selectively retrieve, and dynamically update personal experiences. The authors propose a multi-resolution memory substrate organized along two axes: representation (structured records, vector embeddings, and graph-based relations) and time (short-term traces, mid-term abstractions, and long-term semantic commitments). A synchronized structure-vector-graph mechanism enables selective memory retrieval, validation, and integration. Innovatively framing personalized reliability as a memory design problem, the approach emphasizes structured storage, selective exposure, continual integration, and cognitive tagging. A prototype system demonstrates the framework’s capability in pre-generating memory candidates, revising content, enforcing boundary constraints, and tracing evidential provenance, validating its efficacy in extended interaction scenarios.

Current research on agent memory systems is hindered by fragmented architectures, tightly coupled components, evaluation protocols bound to specific datasets, and insufficient support for heterogeneous memory types. This work proposes an interoperable memory research framework that decouples stages of the memory lifecycle through declarative data contracts, separates benchmark datasets from execution protocols, and introduces a unified computational interface to harmonize symbolic, neural, and multimodal memory representations. The framework enables, for the first time, cross-platform plug-and-play memory components, orthogonal separation of evaluation protocols and datasets, and unified runtime coordination of heterogeneous memory types. Experiments demonstrate that the approach facilitates cross-system integration, flexible reconfiguration of evaluation pipelines, and systematic isolation and analysis of memory design variables.

This work addresses the lack of effective long-term memory management in large language model agents during extended interactions. Inspired by human cognition, the authors propose a novel memory architecture that systematically integrates six key mechanisms: sleep-based consolidation, interference-driven forgetting, memory trace maturation, retrieval-induced reconsolidation, entity-centric knowledge graphs, and multi-cue hybrid retrieval. To prevent data leakage, they introduce an unsupervised synthetic calibration method for threshold setting. The framework further incorporates memory deduplication and compression, context budget control, and a streaming multi-level evaluation paradigm. Evaluated on the VSCode dataset, the approach achieves 97.2% memory retention accuracy while reducing storage overhead by 58%. On the LongMemEval benchmark, it matches baseline retrieval accuracy using only 200K tokens of context and improves S-tier preference recall by 13.3 percentage points.

Existing memory systems for large language model agents suffer from high maintenance overhead, poor scalability, and increasing latency as memory grows, primarily due to coarse-grained state management and sequential update mechanisms. This work reframes memory management as a write-efficient time-series data problem and introduces MemTree—a hierarchical temporal indexing structure that replaces global summarization with time-ordered trees, enabling path-localized updates. Additionally, it incorporates parallel chunk extraction and a decoupled memory construction pipeline. The proposed approach substantially reduces maintenance costs while preserving the temporal evolution of states, achieving a 79.8% pass@1 accuracy on LongMemEval-S and a memory construction throughput six times higher than the current best method, such as EverMemOS.

This work addresses the challenge in existing long-term conversational agents where memory systems struggle to balance reasoning efficiency with effective relational modeling—flat memories lack structure, while graph-based approaches incur high construction overhead. To this end, we propose StructMem, a structure-enhanced hierarchical memory framework that leverages temporal anchoring, dual-perspective modeling, and periodic semantic integration to preserve event-level bindings while establishing cross-event associations. By avoiding the explicit construction of costly knowledge graphs, StructMem enables efficient relational reasoning without sacrificing scalability. Experimental results on the LoCoMo dataset demonstrate that our approach significantly improves performance in temporal reasoning and multi-hop question answering, while substantially reducing token consumption, API calls, and runtime.