memory-augmented networks

This work addresses the limited human-like long-term memory capabilities of large language model (LLM) agents, noting that existing semantic similarity–based memory mechanisms fail to capture the associative structure inherent in human memory. To bridge this gap, the authors propose a cognitive neuroscience–inspired dynamic graph memory architecture that constructs episodic memory graphs via Hebbian learning and integrates a reflective agent to distill frequently activated nodes into structured semantic knowledge, thereby enabling the co-evolution of episodic and semantic memory. This approach uniquely incorporates Hebbian learning, memory consolidation, and spreading activation into LLM memory systems, supporting associative retrieval and knowledge distillation. Experimental results demonstrate superior performance across all four question categories of the LoCoMo benchmark while significantly reducing context token consumption.

This work addresses the challenge of balancing stability and plasticity in continual learning under sequential data scenarios. Inspired by the modular organization of the human brain, the authors propose MoRe, a novel framework that constructs a theoretically identifiable hierarchical modular structure in representation space. MoRe decomposes knowledge into shared foundational modules and task-specific modules, enabling module reuse, alignment, and expansion. By leveraging temporal delayed dependencies to uncover intrinsic sequence structures and integrating modular learning with identifiability constraints, MoRe achieves structured knowledge organization and protection without requiring explicit task boundaries. Experiments on synthetic benchmarks and activation data from large language models demonstrate that MoRe learns interpretable hierarchical representations and significantly improves the stability-plasticity trade-off in continual learning.

This work addresses the construction of human-like memory mechanisms in large language models and multimodal foundation models to support continual learning, personalized reasoning, and cross-modal consistency. It proposes the first unified taxonomic framework that systematically integrates three major memory paradigms: implicit memory (e.g., parameterized memory), explicit memory (e.g., external retrieval and graph-structured knowledge bases), and agent memory. The framework is further extended to multimodal settings, elucidating the critical role of memory in cross-modal alignment and agent collaboration. Through a comprehensive literature review and taxonomic analysis, the study surveys existing technical approaches, evaluation benchmarks, and core challenges, thereby establishing a theoretical foundation and offering clear directions for future research on human-like memory systems in artificial intelligence.

Existing retrieval-augmented approaches struggle to handle the fragmented nature of long-term agent memory and inadequately support complex temporal and multi-hop reasoning tasks. This work proposes a dynamic schematic memory architecture that emulates the spreading activation mechanism from cognitive science, dynamically selecting relevant subgraphs through lateral inhibition and temporal decay to enable synergistic retrieval of semantic and episodic memory. By moving beyond static vector similarity, the model mitigates the “context tunneling” problem and integrates geometric embedding with activation-driven graph traversal to form a tripartite hybrid retrieval strategy. Evaluated on the LoCoMo benchmark, the proposed method significantly outperforms current state-of-the-art approaches, demonstrating superior performance in complex reasoning scenarios.

Large language models (LLMs) exhibit limited in-context learning (ICL) capabilities compared to biological neural systems. Method: Inspired by associative memory mechanisms in neuroscience, we propose a novel attention residual flow architecture that introduces cross-layer direct connections between attention heads and a head-value focusing mechanism to enable efficient information propagation and integration along residual paths. Contribution/Results: This work is the first to incorporate associative memory modeling into Transformer residual design. Evaluated on a compact two-layer model with only 8M parameters, our architecture demonstrates earlier emergence of ICL ability and achieves significantly higher accuracy across multiple standard ICL benchmarks. These results indicate that the proposed architecture enhances contextual generalization in small-scale models under more biologically plausible computational constraints, establishing a new brain-inspired paradigm for ICL modeling.

This work addresses the challenge of jointly achieving domain generalization and continual learning in multi-task settings, where catastrophic forgetting commonly occurs. We propose MIRA, a unified framework that integrates Hopfield-style associative memory into adapter architectures. Its core innovation lies in a reconfigurable shared backbone coupled with sample-level dynamic retrieval, enabled by post-training key learning and affine composition-based updates for task-adaptive modulation. This design emulates neuromodulatory regulation of a single neural circuit, facilitating rapid task switching and persistent knowledge retention. On standard benchmarks, MIRA achieves state-of-the-art out-of-distribution accuracy for domain generalization while significantly outperforming dedicated continual learning methods. Notably, it demonstrates superior knowledge retention across incremental tasks, effectively mitigating catastrophic forgetting without compromising generalization.

This work addresses the lack of a dynamic theoretical framework in existing associative memory models for sequential retrieval and multi-memory integration, which hinders their ability to capture sequential reasoning mechanisms in modern machine learning. The paper proposes an Input-Driven Plasticity (IDP) Hopfield network featuring a dual-timescale architecture that couples fast associative retrieval with slow inferential dynamics, offering the first rigorous mathematical characterization of sequentiality in associative memory. Through nonlinear dynamical and stability analyses, the authors derive explicit conditions for self-sustained memory transitions—including gain thresholds, escape times, and collapse regions—thereby establishing a theoretical bridge between classical Hopfield dynamics and contemporary inference architectures, and providing a verifiable foundation for sequential reasoning.

Existing sequence models struggle to emulate human memory consolidation mechanisms, limiting their ability to efficiently process long contexts during inference while preserving multi-granular memory representations. Inspired by the neuroscientific hypothesis of memory transformation, this work proposes Mela, a novel architecture that employs a Hierarchical Memory Module (HMM) to disentangle high-frequency details from low-frequency abstract information. By integrating cross-frequency coupling and a MemStack feature distribution strategy, Mela enables online, hierarchical memory consolidation without increasing token count. To our knowledge, this is the first approach to incorporate human-like memory consolidation into sequence modeling. Integrated into a Transformer decoder, Mela significantly outperforms baseline models under a fixed 4K pretraining context length and maintains stable performance even when evaluated on substantially longer test sequences.

Current research on memory mechanisms in large language models remains highly fragmented and lacks a unified theoretical framework. This work proposes an architecture-centered taxonomy that systematically models memory along three orthogonal dimensions: representation, update dynamics, and persistence, formally characterizing core processes such as writing, routing, state transition, and integration. By constructing the first three-dimensional unified framework that integrates implicit/explicit, offline/online, and short-term/long-term memory, it clarifies the boundary between computationally coupled memory and independently addressable memory. Through a systematic literature review, architectural analysis, and multidimensional evaluation, the study synthesizes techniques including attention mechanisms, recurrent states, parameter-efficient fine-tuning, and scalable retrieval-augmented storage, thereby establishing a coherent paradigm for memory modeling and providing a theoretical foundation and design principles for future scalable and adaptive large language models.

This work addresses the limitations of traditional retrieval-augmented generation (RAG) in personalized AI assistants, which suffer from constrained memory capacity and insufficient integration with dialogue history, often leading to redundant clarifications and irrelevant document retrieval. Inspired by the hippocampal–neocortical memory system, we propose an end-to-end architecture that integrates short-term and long-term memory mechanisms. Our approach employs reinforcement learning to compress dialogue context and pre-retrieve relevant knowledge, introduces a short-term memory extractor that mimics hippocampal–prefrontal coordination, and constructs a partitioned long-term memory network to store user-specific information and rerank retrieved results. Evaluated on a real-world industrial dataset, the proposed method significantly outperforms cascaded RAG baselines, and ablation studies confirm the effectiveness and necessity of both memory components.