design category-equivariant networks

This work addresses the insufficient modeling of complex symmetric structures in deep learning—such as non-invertible symmetries and higher-order relations beyond graphs—by introducing Order-Equivariant Neural Networks (OENN) and Category-Equivariant Neural Networks (CENN). Grounded in equivariant bundles, face posets, and category theory, this study provides the first complete characterization of all linear order-equivariant maps and establishes a Universal Approximation Theorem (UAT), thereby filling a critical theoretical gap: the absence of a UAT for layered neural architectures. Furthermore, it generalizes the UAT for graph neural networks to a broader equivariant framework. The proposed architectures unify message-passing mechanisms across graph and layered models, demonstrating empirical effectiveness and offering a cohesive theoretical foundation with guaranteed approximation capabilities for equivariant deep learning.

This work addresses the lack of explicit symmetry modeling in standard neural networks. We propose “equivarification,” a general equivariance-enabling framework that transforms arbitrary off-the-shelf architectures into models equivariant to user-specified groups (e.g., SE(2))—without architectural modification. Guided by group representation theory, the method applies tensor rearrangement, feature-space projection, and symmetry-constrained convolutional kernels. Crucially, it achieves plug-and-play equivariance for generic network designs. Evaluated on CNN-based image classification, equivarified models demonstrate significantly improved robustness to rotations and translations, validating both efficacy and generalizability. Our core contribution lies in bridging generic neural architectures with symmetry priors: we rigorously embed group-equivariant inductive biases while fully preserving architectural flexibility and expressivity.

Equivariant networks strictly preserve input symmetries, rendering them ill-suited for generative tasks requiring *active symmetry breaking*—e.g., reconstructing asymmetric structures from highly symmetric latent representations. To address this, we establish the first necessary and sufficient representation theorem for equivariant conditional distributions and propose SymPE: a method that achieves *controllable symmetry breaking* via learnable stochastic normalized positional encodings, while preserving the group-equivariant inductive bias. SymPE unifies probabilistic symmetry breaking, positional encoding, and equivariant graph neural networks, and naturally integrates with diffusion-based generative frameworks. Empirically, it significantly improves performance on graph diffusion modeling, graph autoencoding, and lattice spin system generation. Theoretically, we prove that SymPE’s generalization bound is strictly superior to that of conventional equivariant networks.

This work addresses the challenge of modeling permutation-equivariant representations for unaligned symmetric sets—such as unordered point clouds and graph node sets—by systematically characterizing linear layers equivariant to the symmetric group and its extension, the cyclic product group. Leveraging group representation theory, irreducible decomposition, and Schur’s lemma, we provide a unified reconstruction of DeepSets, 2-IGN, and DWS, and present the first complete classification of fully order-equivariant linear layers under the cyclic product group, revealing numerous novel non-Siamese architectures. Our framework significantly simplifies DWS derivation and removes restrictive alignment assumptions. Empirically, the proposed non-Siamese equivariant layers achieve improved performance on graph anomaly detection, neural network weight-space alignment, and Wasserstein distance learning. Code is publicly available.

Random neural networks lack principled frameworks for structured symmetry modeling. Method: We propose the first general equivariant symmetrization paradigm for stochastic models. Within the Markov category framework, we unify deterministic and stochastic mappings; leveraging group representation theory and categorical semantics, we systematically lift H-equivariant networks to G-equivariant ones via group homomorphisms, enabling composable symmetrization. We further extend classical averaging to the stochastic setting under minimally restrictive assumptions, ensuring broad theoretical applicability. Contribution/Results: We establish the first composable, mathematically rigorous theory for symmetrizing stochastic neural networks—overcoming the longstanding limitation that equivariance theory applies only to deterministic models. Our framework significantly enhances structural expressivity and improves generalization with greater interpretability, providing a foundational advance for equivariant learning in probabilistic settings.

This work investigates the construction of convolutional neural networks on smooth manifolds that exhibit equivariance under Lie groupoids and Lie algebroids. To this end, the authors propose a unified network architecture comprising Lie groupoid-lifted convolutions, Lie algebroid-equivariant layers, and groupoid-invariant global pooling, all formulated as natural transformations between continuous feature functors. This study establishes, for the first time, a systematic theoretical framework for Lie groupoid- and Lie algebroid-equivariant neural networks, demonstrating that each proposed component arises as a special case of admissible categorical equivariant layers. Furthermore, under suitable conditions, the equivalence between these two classes of networks is rigorously proven, thereby successfully extending topological categorical equivariance methods to the setting of differential geometry.

This work proposes a unified categorical framework for modeling symmetries and equivariant representations in deep learning through a coalgebraic formalism. By embedding invariant structures from the category of sets into the category of vector spaces via functors, and constructing compatible endofunctors to capture equivariance, the paper systematically introduces coalgebraic tools to bridge abstract symmetry specifications with concrete neural network implementations. Within this framework, the authors establish a universal approximation theorem for continuous equivariant functions, applicable to a broad class of symmetry groups. This result provides a rigorous theoretical foundation and realizability guarantee for equivariant deep learning architectures, thereby advancing the principled design of symmetry-aware neural models.

This work addresses the issue of symmetry amplification in equivariant neural networks, which can impair their ability to represent geometric structures when processing symmetric inputs. The study provides the first rigorous proof that this phenomenon is bounded below by a limit determined by the structure of the feature space. Leveraging group representation theory and an analysis of equivariant mappings, the authors develop a computable framework to determine this infimum. Based on this theoretical foundation, they propose design principles for constructing features that mitigate detrimental symmetry amplification. Experiments on synthetic data and the QM9 dataset validate the theoretical predictions, demonstrating that the proposed approach effectively alleviates symmetry amplification and enhances model expressivity.

Existing equivariant networks typically support only fixed symmetry groups, limiting their ability to flexibly handle multimodal data with diverse symmetries. This work proposes the ASEN model, which achieves simultaneous equivariance to multiple permutation subgroups within a single architecture by incorporating symmetry-breaking auxiliary input features and leveraging an approximate symmetry-breaking mechanism together with an efficient 2-closure fast algorithm. Built upon a fully permutation-equivariant basis model and employing equivariant MLP emulation techniques, ASEN overcomes the rigidity of conventional equivariant networks. Experiments demonstrate that ASEN outperforms both specialized equivariant models and non-equivariant baselines across tasks involving graph and image symmetry selection, as well as sequence-based multitask and transfer learning scenarios.

This work addresses the limitation of conventional meta-networks, which rely solely on raw parameters and overlook the intrinsic symmetries of neural architectures, thereby struggling to capture functional equivalence. To overcome this, the authors propose quasi-equivariant meta-networks, introducing a novel paradigm of “quasi-equivariance” that relaxes strict equivariance constraints while preserving functional identity. This approach strategically balances architectural symmetry with model expressivity through group actions and a relaxed equivariance mechanism, making it compatible with a variety of mainstream network architectures. Empirical results demonstrate that quasi-equivariant meta-networks consistently achieve a superior trade-off between symmetry preservation and representational capacity across multiple architectures, significantly outperforming existing strictly equivariant methods.