🤖 AI Summary
This work addresses the challenge of modeling permutation-equivariant representations for unaligned symmetric sets—such as unordered point clouds and graph node sets—by systematically characterizing linear layers equivariant to the symmetric group and its extension, the cyclic product group. Leveraging group representation theory, irreducible decomposition, and Schur’s lemma, we provide a unified reconstruction of DeepSets, 2-IGN, and DWS, and present the first complete classification of fully order-equivariant linear layers under the cyclic product group, revealing numerous novel non-Siamese architectures. Our framework significantly simplifies DWS derivation and removes restrictive alignment assumptions. Empirically, the proposed non-Siamese equivariant layers achieve improved performance on graph anomaly detection, neural network weight-space alignment, and Wasserstein distance learning. Code is publicly available.
📝 Abstract
This paper explores the characterization of equivariant linear layers for representations of permutations and related groups. Unlike traditional approaches, which address these problems using parameter-sharing, we consider an alternative methodology based on irreducible representations and Schur's lemma. Using this methodology, we obtain an alternative derivation for existing models like DeepSets, 2-IGN graph equivariant networks, and Deep Weight Space (DWS) networks. The derivation for DWS networks is significantly simpler than that of previous results. Next, we extend our approach to unaligned symmetric sets, where equivariance to the wreath product of groups is required. Previous works have addressed this problem in a rather restrictive setting, in which almost all wreath equivariant layers are Siamese. In contrast, we give a full characterization of layers in this case and show that there is a vast number of additional non-Siamese layers in some settings. We also show empirically that these additional non-Siamese layers can improve performance in tasks like graph anomaly detection, weight space alignment, and learning Wasserstein distances. Our code is available at href{https://github.com/yonatansverdlov/Irreducible-Representations-of-Deep-Weight-Spaces}{GitHub}.