representation theory

This work investigates whether partial data augmentation can statistically match the generalization performance and sample complexity of full-group augmentation under computational constraints. By leveraging Fourier analysis and finite group representation theory, the authors establish a unified theoretical framework that, for the first time, characterizes the conditions under which partial and full augmentations are statistically equivalent from a frequency-domain perspective. The main contributions include proving that when the augmented subset is sufficiently large, partial augmentation achieves the same minimax optimal rate as full augmentation, while also demonstrating that exact symmetry—leading to perfect invariance—can only be realized through averaging over the entire group. Consequently, the study delineates the theoretical limits of approximate symmetry and establishes an impossibility result showing that no proper subgroup can yield exact invariance.

This study investigates whether narrow neural networks can spontaneously learn abstract algebraic structures—such as identity elements, commutativity, and subgroups—from group operations without explicit supervision. By training small networks to predict group operations (e.g., modular addition or permutation composition) and employing linear probing, representational analysis, and structural tests, the work provides the first systematic evaluation of how well internal representations implicitly capture algebraic concepts. The results demonstrate that models effectively encode commutativity and enable linear classifiers to accurately identify elements of specific subgroups, yet exhibit limited capacity to learn notions like the identity element. This research introduces a novel paradigm for extracting mathematical structures from neural networks and reveals both the potential and limitations of narrow architectures in abstract algebraic reasoning.

This work investigates how to directly learn algebraic properties of finite groups—such as commutativity, nilpotency, and solvability—from their Cayley graphs. To this end, we propose the first unified graph neural network (GNN) framework capable of end-to-end extraction of algebraic structure from Cayley graphs without requiring property-specific model customization. Employing a general-purpose GNN architecture and training protocol, our approach accurately predicts diverse algebraic properties across multiple families of finite groups. The results demonstrate that Cayley graph representations inherently encode rich algebraic information and establish a novel paradigm at the intersection of group theory and deep learning.

This work investigates how task-relevant symmetries—exact or approximate equivariance—affect the generalization of deep learning models, particularly under symmetry mismatch between model and data. Method: We develop the first generalization bound that does not assume group structure, rigorously quantifying the interplay between model equivariance error and data equivariance error. Our approach integrates probabilistic generalization theory, function approximation theory, and symmetry metrics, accommodating non-group, non-exact, and non-global equivariance settings. Contributions/Results: We establish that precise modeling of task symmetries significantly improves generalization. We formally characterize the optimal error trade-off under approximate or local equivariance when model and data symmetries are misaligned. Furthermore, we derive an “error alignment” principle—a concrete, actionable theoretical guideline for designing robust equivariant models—thereby bridging abstract symmetry considerations with practical architectural design.

This paper investigates whether deep networks under standard supervised training can autonomously learn unencoded symmetries—such as rotation invariance—from partially observed cyclic group-symmetric data, under realistic class-level symmetry heterogeneity (where some classes exhibit full cyclic transformations while others only subsets). Method: We develop the first neural-kernel symmetry learning theory, grounded in infinite-width NTK analysis and group representation-theoretic Fourier analysis. Contribution/Results: The theory reveals that generalization to unseen symmetries hinges on the “overwhelming dominance” of local data structure over symmetry structure within the kernel-induced feature space, and yields a verifiable signal-to-noise criterion in the frequency domain. Our analysis precisely reproduces empirical failures of MLPs, CNNs, and ViTs on rotationally augmented MNIST subsets, and rigorously proves that conventional supervised training cannot acquire symmetries absent from the architectural prior.

This work investigates the application of graph neural networks (GNNs) to determine the solvability of finite groups. Addressing this fundamental problem in abstract algebra, we propose the first approach that leverages GNNs on graph representations of finite groups—such as Cayley graphs—to learn from their structural properties and predict solvability. Experimental results demonstrate that the proposed model effectively distinguishes between solvable and non-solvable groups even on out-of-distribution instances not seen during training. These findings confirm that GNNs can capture deep algebraic properties inherent in group structures, thereby establishing a novel paradigm for integrating geometric representations of algebraic objects with machine learning techniques.

This study investigates the fundamental mechanisms underlying the "grokking" phenomenon—characterized by high training accuracy coupled with delayed generalization—in feature-learning kernels, with a focus on the role of data symmetry. Employing Recursive Feature Machines (RFMs) and iteratively updating feature matrices via the Average Gradient Outer Product (AGOP), the authors analyze grokking behavior in algebraic tasks. Their central finding is that generalization occurs only when the symmetry of the training data is broken. The RFM achieves generalization by recovering the intrinsic group action governing the data, with the learned feature matrix precisely encoding the structure of this symmetry group. This work provides the first empirical evidence that symmetry breaking is a necessary condition for generalization and elucidates the group-theoretic underpinnings of grokking.

This study investigates how to effectively learn the algebraic structure and symmetry properties of finite groups from their Cayley graphs. We construct a large-scale dataset comprising 131,406 Cayley graphs, covering all finite groups of order up to 767 (excluding 512), thereby establishing for the first time a systematic correspondence between graph structure and group-theoretic properties. Using graph neural networks (GIN, GCN), multilayer perceptrons, and handcrafted graph statistics—such as cycle counts, distance distributions, and spectral features—we predict various group properties. Our results reveal that handcrafted features are highly informative, and GNNs effectively recover structural signals of the underlying groups, significantly outperforming conventional models on specific tasks. The work demonstrates the capacity of graph representations to encode group symmetries, proposes several empirically verifiable patterns, and contributes multiple new integer sequences to the OEIS.

This work investigates how small Transformer models learn modular integer multiplication under composite moduli, where the operation is not globally invertible. To this end, the authors introduce a “monoid extension” framework that partitions the input space into local algebraic regions, each preserving a group-like structure and incorporating a localized Fourier mechanism. This approach constitutes the first extension of representation-theoretic Fourier analysis to non-group algebraic structures containing zero divisors. Through a combination of representation theory, attention routing analysis, and identification of low-rank writing directions, the study reveals that in square-free modulus multiplication tasks, embeddings are organized by algebraic region, attention exhibits class-sensitive routing, and local Fourier features account for the majority of output logits.

Determining whether a subgroup isomorphism exists between finite groups is a fundamental problem in computational group theory. This work proposes a novel unified learning framework that integrates geometric deep learning with algebraic features: group structures are represented via Cayley graphs, and a siamese graph neural network generates corresponding graph embeddings; these are then combined with handcrafted algebraic invariants to form a joint feature vector, which is fed into a fully connected classifier to predict subgroup relationships. Evaluated on an independent test set, the method achieves an accuracy of 95.9% (47 out of 49 cases), strongly demonstrating the effectiveness and innovation of the proposed framework.