🤖 AI Summary
This work addresses the insufficient modeling of complex symmetric structures in deep learning—such as non-invertible symmetries and higher-order relations beyond graphs—by introducing Order-Equivariant Neural Networks (OENN) and Category-Equivariant Neural Networks (CENN). Grounded in equivariant bundles, face posets, and category theory, this study provides the first complete characterization of all linear order-equivariant maps and establishes a Universal Approximation Theorem (UAT), thereby filling a critical theoretical gap: the absence of a UAT for layered neural architectures. Furthermore, it generalizes the UAT for graph neural networks to a broader equivariant framework. The proposed architectures unify message-passing mechanisms across graph and layered models, demonstrating empirical effectiveness and offering a cohesive theoretical foundation with guaranteed approximation capabilities for equivariant deep learning.
📝 Abstract
Symmetry is everywhere in nature and society. Geometric deep learning exploits symmetries in data to improve the performance and efficiency of deep learning systems. In this paper, we extend geometric deep learning to utilize richer symmetry structures. Specifically, we develop order-equivariant neural networks (OENN), which generalize standard graph message passing and sheaf neural networks via the theory of equivariant bundles over face posets (face categories). We (i) characterize all linear order-equivariant maps, (ii) build OENN layers, and (iii) prove universal approximation theorems (UATs) for continuous order-equivariant maps, which are new results even when restricted to sheaf neural networks (for which no UAT was known before). We illustrate the framework on graph and sheaf models. Our results can also be seen as extending the known UAT for graph neural networks to a more general setting that subsumes sheaf neural networks as well. In addition, we show that OENN can be extended further to CENN, Category-Equivariant Neural Network, which gives the general form of equivariant neural networks as well as of equivariant universal approximation theorems, allowing us to leverage categorical symmetry in data (e.g., non-invertible symmetries on multiple objects with compositional relations on those symmetries).