Improving Continual Learning of Knowledge Graph Embeddings via Informed Initialization

2025-11-14

AI Summary

In continual knowledge graph (KG) learning, two key challenges persist: poor initialization of embeddings for newly introduced entities and severe catastrophic forgetting. To address these, this paper proposes an informed initialization method leveraging both KG schema information and historical entity embeddings. Specifically, the method integrates semantic cues from entity types, the geometric distribution of existing embeddings, and structural constraints derived from the KG schema to generate initial embeddings that are semantically coherent and geometrically compatible with the learned embedding space. Compared to standard random or zero-initialization strategies, this approach significantly accelerates adaptation to new knowledge while mitigating performance degradation on previously learned relations. Extensive experiments across prominent knowledge graph embedding (KGE) models—including TransE, RotatE, and ComplEx—demonstrate that our method reduces training epochs by 30% on average, improves inference accuracy by 2.1–5.7 percentage points, and sustains superior knowledge retention under long-term incremental learning. The approach is model-agnostic and practically deployable.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Machine Learning: Structured Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Many Knowledege Graphs (KGs) are frequently updated, forcing their Knowledge Graph Embeddings (KGEs) to adapt to these changes. To address this problem, continual learning techniques for KGEs incorporate embeddings for new entities while updating the old ones. One necessary step in these methods is the initialization of the embeddings, as an input to the KGE learning process, which can have an important impact in the accuracy of the final embeddings, as well as in the time required to train them. This is especially relevant for relatively small and frequent updates. We propose a novel informed embedding initialization strategy, which can be seamlessly integrated into existing continual learning methods for KGE, that enhances the acquisition of new knowledge while reducing catastrophic forgetting. Specifically, the KG schema and the previously learned embeddings are utilized to obtain initial representations for the new entities, based on the classes the entities belong to. Our extensive experimental analysis shows that the proposed initialization strategy improves the predictive performance of the resulting KGEs, while also enhancing knowledge retention. Furthermore, our approach accelerates knowledge acquisition, reducing the number of epochs, and therefore time, required to incrementally learn new embeddings. Finally, its benefits across various types of KGE learning models are demonstrated.

Problem

• Initializing embeddings for new entities in evolving knowledge graphs effectively
• Reducing catastrophic forgetting while acquiring new knowledge in continual learning
• Accelerating training time and improving predictive performance of embeddings

Innovation

• Informed initialization strategy for embedding new entities
• Utilizes KG schema and prior embeddings for initialization
• Enhances knowledge acquisition and reduces catastrophic forgetting