Toward a Unified Statistical Theory of Unsupervised Pretraining and Supervised Neural Knowledge Graph Learning

2026-07-28

AI Summary

This work addresses the scarcity of relation-specific labeled triples and the lack of theoretical grounding in scoring functions within knowledge graph representation learning. To this end, the authors propose a two-stage end-to-end training framework that first performs unsupervised pretraining on heterogeneous corpora and then integrates supervised learning across multiple relation types to jointly model representation and reasoning. For the first time, they establish a non-asymptotic risk bound grounded in statistical learning theory, theoretically decoupling the error from pretrained representations and the sample complexity of labeled data, thereby quantifying the benefit of unlabeled data for downstream prediction. The approach accommodates general scoring architectures and diverse relation types, demonstrating both theoretical validity and significant performance gains on synthetic and large-scale benchmark datasets.

Abstract

Knowledge graph learning provides a powerful framework for representing and inferring structured knowledge, with broad practical applications. However, the scarcity of relation-specific labeled triples per entity hinders the training of expressive models, and the ad hoc design of scoring functions limits generalizability and lacks theoretical grounding. We address both issues with a theoretically grounded, end-to-end training framework that extends and subsumes existing methods. Our framework is a two-stage procedure: unsupervised pretraining over heterogeneous corpora followed by supervised learning with multiple relation types. We establish a nonasymptotic risk bound that disentangles pretraining representation error from labeled-sample complexity, formally quantifying the benefit of large-scale unlabeled data for downstream knowledge prediction. Synthetic experiments validate each theoretical component, and real-world experiments confirm the effectiveness of our approach on large-scale knowledge graph benchmarks.

Problem

• knowledge graph learning
• data scarcity
• scoring functions
• theoretical grounding
• generalizability

Innovation

• unsupervised pretraining
• knowledge graph learning
• nonasymptotic risk bound
• representation error
• theoretical unification