Graph Memory Learning: Imitating Lifelong Remembering and Forgetting of Brain Networks

2024-07-27

IEEE Transactions on Pattern Analysis and Machine Intelligence

AI Summary

Real-world graph data continuously evolve dynamically, necessitating privacy-preserving data unlearning—yet existing models struggle to balance efficient incremental learning with controllable forgetting. To address this, we propose a novel graph memory learning paradigm and introduce the Brain-inspired Graph Memory Learning (BGML) framework—the first to unify incremental learning and reverse forgetting via multi-granularity hierarchical progressive learning and self-assessing information attribution. BGML integrates brain network dynamics modeling, functional-structural coupling, feature-map-granular learning, and incremental topology self-calibration. We further establish a comprehensive benchmark comprising five graph memory tasks. Extensive experiments on multiple node classification datasets demonstrate that BGML significantly outperforms state-of-the-art methods in both data/class-incremental learning and proactive unlearning scenarios, while maintaining high accuracy and preserving historical knowledge integrity.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Graph data in real-world scenarios undergo rapid and frequent changes, making it challenging for existing graph models to effectively handle the continuous influx of new data and accommodate data withdrawal requests. The approach to frequently retraining graph models is resource intensive and impractical. To address this pressing challenge, this paper introduces a new concept of graph memory learning. Its core idea is to enable a graph model to selectively remember new knowledge but forget old knowledge. Building on this approach, the paper presents a novel graph memory learning framework - Brain-inspired Graph Memory Learning (BGML), inspired by brain network dynamics and function-structure coupling strategies. BGML incorporates a multi-granular hierarchical progressive learning mechanism rooted in feature graph grain learning to mitigate potential conflict between memorization and forgetting in graph memory learning. This mechanism allows for a comprehensive and multi-level perception of local details within evolving graphs. In addition, to tackle the issue of unreliable structures in newly added incremental information, the paper introduces an information self-assessment ownership mechanism. This mechanism not only facilitates the propagation of incremental information within the model but also effectively preserves the integrity of past experiences. We design five types of graph memory learning tasks: regular, memory, unlearning, data-incremental, and class-incremental to evaluate BGML. Its excellent performance is confirmed through extensive experiments on multiple node classification datasets.

Problem

• Handling continuous new data and withdrawal requests in evolving graphs
• Avoiding resource-intensive retraining of graph models on changing data
• Balancing selective memorization of new knowledge while forgetting old information

Innovation

• Brain-inspired hierarchical progressive learning mechanism
• Information self-assessment ownership for reliable propagation
• Multi-granular feature learning to resolve memory conflicts