Are Common Substructures Transferable? Riemannian Graph Foundation Model with Neural Vector Bundles

2026-06-02

AI Summary

This work addresses the lack of theoretical foundations for substructure transferability in graph data by bridging transferable substructures with the intrinsic geometry of graph representation spaces from a functional behavior perspective. It proposes the first Riemannian geometry–based framework for learning intrinsic graph geometry, innovatively introducing neural vector bundles and local coordinate charts to construct the GAUGE pretraining architecture. A Dirichlet loss function is designed to enable explicit modeling of intrinsic graph geometry and quantification of transfer difficulty. The method demonstrates significant performance gains over existing models on zero-shot link prediction and graph isomorphism tasks, validating its expressive power and cross-task transferability.

Abstract

Foundation models have sparked a revolution via a pretraining-adaptation paradigm, with recent efforts extending this success to graphs. Unlike other modalities, graphs contain rich structural patterns, yet their structural transferability remains poorly understood. Prior studies consider common substructures in the discrete realm, and we are motivated by a fundamental question: Are common substructures transferable? The underlying theory is largely underexplored. In this work, we shift toward learning transferable structures through the lens of functional behavior. Theoretically, we connect transferable substructures to intrinsic geometry of the representation space. However, characterizing such intrinsic geometry has rarely been touched. Grounded in Riemannian geometry, we develop a graph intrinsic geometry learning framework called Neural Vector Bundle, which enables parsing intrinsic geometry with local coordinates. Building on this, we design GAUGE, a pretrainable neural architecture that constructs the vector bundle, flattening geometrically compatible local coordinates, and a new Dirichlet loss, which also measures the transfer effort. We empirically validate its superior expressiveness in challenging tasks including zero-shot link prediction and graph isomorphism.

Problem

• graph foundation model
• transferable substructures
• Riemannian geometry
• intrinsic geometry
• structural transferability

Innovation

• Riemannian geometry
• Neural Vector Bundle
• graph foundation model
• transferable substructures
• intrinsic geometry