design equivariant models

Existing equivariant networks typically support only fixed symmetry groups, limiting their ability to flexibly handle multimodal data with diverse symmetries. This work proposes the ASEN model, which achieves simultaneous equivariance to multiple permutation subgroups within a single architecture by incorporating symmetry-breaking auxiliary input features and leveraging an approximate symmetry-breaking mechanism together with an efficient 2-closure fast algorithm. Built upon a fully permutation-equivariant basis model and employing equivariant MLP emulation techniques, ASEN overcomes the rigidity of conventional equivariant networks. Experiments demonstrate that ASEN outperforms both specialized equivariant models and non-equivariant baselines across tasks involving graph and image symmetry selection, as well as sequence-based multitask and transfer learning scenarios.

While equivariant neural networks excel on symmetric tasks, their training is often hampered by optimization difficulties—yet it remains unclear whether the root cause lies in the equivariance constraints themselves or inadequate hyperparameter tuning. Method: We theoretically establish that intrinsic parameter symmetries in unconstrained models strictly impede convergence to the globally optimal solution within the equivariant subspace. To address this, we propose *dynamic group representation relaxation*: instead of enforcing fixed standard equivariant structures, we adaptively reselect group representations at hidden layers based on the optimization trajectory. Contribution/Results: Leveraging group representation theory and loss landscape geometry, we provide the first rigorous proof that symmetry-induced degeneracies obstruct optimization. Empirical validation confirms that relaxed weights indeed correspond to distinct group representations. Our work establishes a verifiable geometric principle for training equivariant models, bridging theory and practice in equivariant deep learning.

Equivariant networks strictly preserve input symmetries, rendering them ill-suited for generative tasks requiring *active symmetry breaking*—e.g., reconstructing asymmetric structures from highly symmetric latent representations. To address this, we establish the first necessary and sufficient representation theorem for equivariant conditional distributions and propose SymPE: a method that achieves *controllable symmetry breaking* via learnable stochastic normalized positional encodings, while preserving the group-equivariant inductive bias. SymPE unifies probabilistic symmetry breaking, positional encoding, and equivariant graph neural networks, and naturally integrates with diffusion-based generative frameworks. Empirically, it significantly improves performance on graph diffusion modeling, graph autoencoding, and lattice spin system generation. Theoretically, we prove that SymPE’s generalization bound is strictly superior to that of conventional equivariant networks.

Equivariant neural networks suffer from training difficulties, optimization instability, and hyperparameter sensitivity due to strict equivariance constraints. To address this, we propose a progressive constraint relaxation training framework: learnable non-equivariant compensation terms are introduced in intermediate layers, and their activation is gradually tightened via a dynamically decaying soft regularization term—expanding the optimization landscape and enhancing robustness early in training while preserving exact equivariance at convergence. This work is the first to model hard equivariance constraints as learnable, time-varying soft constraints, unifying architectural flexibility with theoretical rigor. Experiments across state-of-the-art equivariant architectures—including SE(3)-Transformer and E(n)-GNN—demonstrate consistent improvements: +1.8% average test accuracy, reduced training failure rates, and diminished sensitivity to hyperparameters such as learning rate.

Approximate equivariant neural networks suffer from excessive parameter counts and limited flexibility in modeling symmetries. Method: This paper introduces a structured parameterization framework based on group matrices (GMs), unifying low-displacement-rank (LDR) structures for arbitrary finite groups. By integrating group representation theory with structured matrix modeling, the approach naturally encodes approximate equivariance and generalizes fundamental CNN operations beyond cyclic groups. Contribution/Results: The method preserves approximate equivariance while drastically improving parameter efficiency—reducing parameter counts by one to two orders of magnitude compared to state-of-the-art approximate equivariant networks and structured models. It achieves competitive performance across multiple tasks, offering both theoretical unification—bridging symmetry-aware learning and structured linear algebra—and practical computational benefits.

This work addresses the insufficient modeling of complex symmetric structures in deep learning—such as non-invertible symmetries and higher-order relations beyond graphs—by introducing Order-Equivariant Neural Networks (OENN) and Category-Equivariant Neural Networks (CENN). Grounded in equivariant bundles, face posets, and category theory, this study provides the first complete characterization of all linear order-equivariant maps and establishes a Universal Approximation Theorem (UAT), thereby filling a critical theoretical gap: the absence of a UAT for layered neural architectures. Furthermore, it generalizes the UAT for graph neural networks to a broader equivariant framework. The proposed architectures unify message-passing mechanisms across graph and layered models, demonstrating empirical effectiveness and offering a cohesive theoretical foundation with guaranteed approximation capabilities for equivariant deep learning.

This work addresses the issue of symmetry amplification in equivariant neural networks, which can impair their ability to represent geometric structures when processing symmetric inputs. The study provides the first rigorous proof that this phenomenon is bounded below by a limit determined by the structure of the feature space. Leveraging group representation theory and an analysis of equivariant mappings, the authors develop a computable framework to determine this infimum. Based on this theoretical foundation, they propose design principles for constructing features that mitigate detrimental symmetry amplification. Experiments on synthetic data and the QM9 dataset validate the theoretical predictions, demonstrating that the proposed approach effectively alleviates symmetry amplification and enhances model expressivity.

本文提出了一种基于群胚的等变神经网络理论，用于解决在有界域上构建可操控CNN的问题，并通过稀疏操作实现架构，提高了模型准确性。

This work addresses the limited robustness of deep learning models when encountering rare group-symmetric transformations—such as unusual poses, scales, or positions—during inference. To overcome this challenge, the authors propose a novel paradigm that implicitly learns equivariance directly from data without requiring prior knowledge of the transformation group. By learning equivariant operators in a latent space, the method combines the flexibility of conventional neural networks with the structural advantages of explicitly equivariant architectures, enabling effective generalization to unseen symmetric transformations. Experiments on noisy MNIST datasets with rotation and translation demonstrate that the proposed approach significantly outperforms both standard and explicitly equivariant networks in out-of-distribution classification, confirming its efficacy and strong generalization capability.

This work addresses the limitation of conventional meta-networks, which rely solely on raw parameters and overlook the intrinsic symmetries of neural architectures, thereby struggling to capture functional equivalence. To overcome this, the authors propose quasi-equivariant meta-networks, introducing a novel paradigm of “quasi-equivariance” that relaxes strict equivariance constraints while preserving functional identity. This approach strategically balances architectural symmetry with model expressivity through group actions and a relaxed equivariance mechanism, making it compatible with a variety of mainstream network architectures. Empirical results demonstrate that quasi-equivariant meta-networks consistently achieve a superior trade-off between symmetry preservation and representational capacity across multiple architectures, significantly outperforming existing strictly equivariant methods.