T-REGS: Minimum Spanning Tree Regularization for Self-Supervised Learning

2025-10-27

AI Summary

Self-supervised learning (SSL) suffers from two fundamental challenges: dimensional collapse and non-uniform representation distribution. To address these, we propose T-REGS—the first SSL framework to incorporate minimum spanning tree (MST) length as a geometric-structural regularizer. By explicitly optimizing the geometry and topology of the representation space, T-REGS simultaneously mitigates both issues. We theoretically prove that, on any compact Riemannian manifold, MST-length regularization jointly improves dimensional utilization efficiency and representation uniformity. The method requires no auxiliary network modules or labels; it constructs an MST directly from feature embeddings and minimizes its total edge length—making it inherently compatible with mainstream SSL paradigms. Extensive experiments on synthetic data and standard benchmarks (e.g., ImageNet, CIFAR) demonstrate that T-REGS consistently boosts linear evaluation accuracy and downstream transfer performance, validating the effectiveness and generalizability of geometry-driven regularization in SSL.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Computer Vision: Representation Learning for Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Self-supervised learning (SSL) has emerged as a powerful paradigm for learning representations without labeled data, often by enforcing invariance to input transformations such as rotations or blurring. Recent studies have highlighted two pivotal properties for effective representations: (i) avoiding dimensional collapse-where the learned features occupy only a low-dimensional subspace, and (ii) enhancing uniformity of the induced distribution. In this work, we introduce T-REGS, a simple regularization framework for SSL based on the length of the Minimum Spanning Tree (MST) over the learned representation. We provide theoretical analysis demonstrating that T-REGS simultaneously mitigates dimensional collapse and promotes distribution uniformity on arbitrary compact Riemannian manifolds. Several experiments on synthetic data and on classical SSL benchmarks validate the effectiveness of our approach at enhancing representation quality.

Problem

• Prevents dimensional collapse in self-supervised learning
• Promotes uniformity of learned feature distributions
• Regularizes representations using minimum spanning tree length

Innovation

• Regularizes SSL with Minimum Spanning Tree length
• Mitigates dimensional collapse in learned representations
• Promotes uniformity of distribution on manifolds