Learning Symmetries via Weight-Sharing with Doubly Stochastic Tensors

2024-12-05

arXiv.org

AI Summary

This work addresses the challenge of modeling unknown or partially symmetric data in deep learning. We propose an adaptive group-equivariant learning framework that eliminates reliance on predefined symmetry groups. Our core method introduces a learnable doubly stochastic matrix as a soft permutation operator, unifying explicit group structures and implicit partial symmetries; it further integrates differentiable weight sharing, soft kernel transformations, and joint task-driven optimization to dynamically construct soft weight-sharing structures. Theoretically and empirically, our approach converges to standard group convolutions on strongly symmetric data, while significantly outperforming fixed-equivariance baselines on partially symmetric data—yielding higher accuracy, improved robustness, enhanced generalization, and greater data efficiency.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Group equivariance has emerged as a valuable inductive bias in deep learning, enhancing generalization, data efficiency, and robustness. Classically, group equivariant methods require the groups of interest to be known beforehand, which may not be realistic for real-world data. Additionally, baking in fixed group equivariance may impose overly restrictive constraints on model architecture. This highlights the need for methods that can dynamically discover and apply symmetries as soft constraints. For neural network architectures, equivariance is commonly achieved through group transformations of a canonical weight tensor, resulting in weight sharing over a given group $G$. In this work, we propose to learn such a weight-sharing scheme by defining a collection of learnable doubly stochastic matrices that act as soft permutation matrices on canonical weight tensors, which can take regular group representations as a special case. This yields learnable kernel transformations that are jointly optimized with downstream tasks. We show that when the dataset exhibits strong symmetries, the permutation matrices will converge to regular group representations and our weight-sharing networks effectively become regular group convolutions. Additionally, the flexibility of the method enables it to effectively pick up on partial symmetries.

Problem

• Symmetry Discovery
• Deep Learning
• Adaptive Learning

Innovation

• Bidirectional Stochastic Tensor
• Automated Symmetry Learning
• Group Equivariance