🤖 AI Summary
To address dimensionality collapse caused by feature aggregation in hyperbolic graph contrastive learning, this paper proposes the first hierarchical contrastive learning framework tailored for the Poincaré ball model. Methodologically, it formally defines uniformity requirements at both leaf-level and height-level hierarchies, and jointly optimizes a hierarchical alignment loss with an isotropic uniformity regularizer—constrained by a differentiable annular density penalty—to overcome limitations of Euclidean contrastive learning paradigms. The contributions are threefold: (1) theoretical modeling of hierarchical uniformity in hyperbolic space; (2) design of a differentiable annular density constraint to mitigate dimensionality collapse; and (3) consistent and significant performance gains across multiple hierarchical graph benchmarks, enhancing both feature space utilization and representation discriminability in downstream tasks.
📝 Abstract
Learning generalizable self-supervised graph representations for downstream tasks is challenging. To this end, Contrastive Learning (CL) has emerged as a leading approach. The embeddings of CL are arranged on a hypersphere where similarity is measured by the cosine distance. However, many real-world graphs, especially of hierarchical nature, cannot be embedded well in the Euclidean space. Although the hyperbolic embedding is suitable for hierarchical representation learning, naively applying CL to the hyperbolic space may result in the so-called dimension collapse, i.e., features will concentrate mostly within few density regions, leading to poor utilization of the whole feature space. Thus, we propose a novel contrastive learning framework to learn high-quality graph embeddings in hyperbolic space. Specifically, we design the alignment metric that effectively captures the hierarchical data-invariant information, as well as we propose a substitute of the uniformity metric to prevent the so-called dimensional collapse. We show that in the hyperbolic space one has to address the leaf- and height-level uniformity related to properties of trees. In the ambient space of the hyperbolic manifold these notions translate into imposing an isotropic ring density towards boundaries of Poincar'e ball. Our experiments support the efficacy of our method.
