e(3)-equivariant film

This work addresses the lack of explicit symmetry modeling in standard neural networks. We propose “equivarification,” a general equivariance-enabling framework that transforms arbitrary off-the-shelf architectures into models equivariant to user-specified groups (e.g., SE(2))—without architectural modification. Guided by group representation theory, the method applies tensor rearrangement, feature-space projection, and symmetry-constrained convolutional kernels. Crucially, it achieves plug-and-play equivariance for generic network designs. Evaluated on CNN-based image classification, equivarified models demonstrate significantly improved robustness to rotations and translations, validating both efficacy and generalizability. Our core contribution lies in bridging generic neural architectures with symmetry priors: we rigorously embed group-equivariant inductive biases while fully preserving architectural flexibility and expressivity.

Although equivariant networks are parameter-efficient, their computational cost rivals that of non-equivariant layers due to the unfolding of structured weights into dense matrices. This work decouples equivariant linear layers into a cyclic convolution over the group dimension and a linear transformation along the channel dimension. Leveraging the Fourier convolution theorem and the conjugate symmetry of the real-valued discrete Fourier transform, we introduce the first efficient frequency-domain acceleration algorithm for such layers. Custom CUDA kernels enable full forward and backward passes in both FP32 and FP16 precision. At the operator level, our method achieves up to 2× speedup over PyTorch’s F.linear, and end-to-end models (Flash EQ-ViT and EQ-Swin) attain up to 1.7× faster inference—marking the first time equivariant networks simultaneously surpass non-equivariant counterparts in accuracy, parameter efficiency, and inference speed.

Approximate equivariant neural networks suffer from excessive parameter counts and limited flexibility in modeling symmetries. Method: This paper introduces a structured parameterization framework based on group matrices (GMs), unifying low-displacement-rank (LDR) structures for arbitrary finite groups. By integrating group representation theory with structured matrix modeling, the approach naturally encodes approximate equivariance and generalizes fundamental CNN operations beyond cyclic groups. Contribution/Results: The method preserves approximate equivariance while drastically improving parameter efficiency—reducing parameter counts by one to two orders of magnitude compared to state-of-the-art approximate equivariant networks and structured models. It achieves competitive performance across multiple tasks, offering both theoretical unification—bridging symmetry-aware learning and structured linear algebra—and practical computational benefits.

Equivariant neural networks suffer from training difficulties, optimization instability, and hyperparameter sensitivity due to strict equivariance constraints. To address this, we propose a progressive constraint relaxation training framework: learnable non-equivariant compensation terms are introduced in intermediate layers, and their activation is gradually tightened via a dynamically decaying soft regularization term—expanding the optimization landscape and enhancing robustness early in training while preserving exact equivariance at convergence. This work is the first to model hard equivariance constraints as learnable, time-varying soft constraints, unifying architectural flexibility with theoretical rigor. Experiments across state-of-the-art equivariant architectures—including SE(3)-Transformer and E(n)-GNN—demonstrate consistent improvements: +1.8% average test accuracy, reduced training failure rates, and diminished sensitivity to hyperparameters such as learning rate.

This work systematically investigates equivariance and invariance of linear neural networks under arbitrary permutation group actions. Method: Leveraging algebraic geometry (determinantal varieties, invariant theory), representation theory of finite groups, and matrix analysis (Eckart–Young theorem), we derive exact geometric characterizations of linear equivariant and invariant function spaces. Contribution/Results: We provide the first complete algebraic characterization of the linear invariant function space under any permutation group—showing it is precisely parameterized by a single linear autoencoder satisfying cycle-decomposition constraints. We further reveal that the equivariant function space exhibits an intrinsic geometric structure: it is multiply connected and admits an irreducible decomposition. Our framework yields structurally simple, implementable parameterizations and corresponding optimization paths. Collectively, these results establish the first rigorous, general, and constructive theoretical foundation for symmetry-driven neural network design.

This work reveals a fundamental limitation imposed by equivariance constraints on the expressive power of two-layer ReLU neural networks: it constructs the first explicit counterexample demonstrating that enforcing equivariance strictly reduces their function representation capacity. Through geometric analysis—examining decision boundary hyperplanes and channel-wise weight vectors—and layer-wise equivariant modeling, the magnitude of this expressivity degradation is quantified. Crucially, the paper proves that modest architectural expansion—e.g., increasing channel width—is sufficient to fully recover the lost expressivity. The key contribution is that the compensated equivariant network retains the original expressive capability while exhibiting a provably lower hypothesis space complexity, yielding a tighter generalization bound. This result provides a theoretical foundation for equivariant neural network design and offers quantitative guidance on the capacity–generalization trade-off.

This work addresses the limitation of conventional meta-networks, which rely solely on raw parameters and overlook the intrinsic symmetries of neural architectures, thereby struggling to capture functional equivalence. To overcome this, the authors propose quasi-equivariant meta-networks, introducing a novel paradigm of “quasi-equivariance” that relaxes strict equivariance constraints while preserving functional identity. This approach strategically balances architectural symmetry with model expressivity through group actions and a relaxed equivariance mechanism, making it compatible with a variety of mainstream network architectures. Empirical results demonstrate that quasi-equivariant meta-networks consistently achieve a superior trade-off between symmetry preservation and representational capacity across multiple architectures, significantly outperforming existing strictly equivariant methods.

This work addresses the issue of symmetry amplification in equivariant neural networks, which can impair their ability to represent geometric structures when processing symmetric inputs. The study provides the first rigorous proof that this phenomenon is bounded below by a limit determined by the structure of the feature space. Leveraging group representation theory and an analysis of equivariant mappings, the authors develop a computable framework to determine this infimum. Based on this theoretical foundation, they propose design principles for constructing features that mitigate detrimental symmetry amplification. Experiments on synthetic data and the QM9 dataset validate the theoretical predictions, demonstrating that the proposed approach effectively alleviates symmetry amplification and enhances model expressivity.

This study investigates the relationship between end-to-end equivariance and layer-wise equivariance in deep neural networks, offering a theoretical explanation for the empirically observed phenomenon wherein weights spontaneously develop equivariant structures during training. Under a parameter identifiability assumption, the work establishes—for the first time—a rigorous proof that if a network as a whole is equivariant under a group action, then there exists a parameter configuration under which each layer is equivariant in its latent space. Leveraging tools from group actions, parameter identifiability, and abstract algebra, the authors construct an architecture-agnostic theoretical framework applicable to a broad class of identifiable networks. This framework provides a solid mathematical foundation for the emergence of equivariant structures in practice and reveals the intrinsic mechanism by which equivariance naturally arises during training.

This work addresses the insufficient modeling of complex symmetric structures in deep learning—such as non-invertible symmetries and higher-order relations beyond graphs—by introducing Order-Equivariant Neural Networks (OENN) and Category-Equivariant Neural Networks (CENN). Grounded in equivariant bundles, face posets, and category theory, this study provides the first complete characterization of all linear order-equivariant maps and establishes a Universal Approximation Theorem (UAT), thereby filling a critical theoretical gap: the absence of a UAT for layered neural architectures. Furthermore, it generalizes the UAT for graph neural networks to a broader equivariant framework. The proposed architectures unify message-passing mechanisms across graph and layered models, demonstrating empirical effectiveness and offering a cohesive theoretical foundation with guaranteed approximation capabilities for equivariant deep learning.