canonical form construction

This paper addresses the unified modeling of symmetries in machine learning. It proposes a framework grounded in differential geometry and convex optimization to (1) enforce known symmetries, (2) automatically discover unknown symmetries in models or data, and (3) actively induce symmetry breaking via user-specified candidate groups. The core contribution is the first formulation of symmetry imposition and discovery as dual linear-algebraic tasks, leveraging the Lie derivative to characterize fiberwise linear Lie group actions on vector bundles, and employing nuclear-norm relaxation to construct convex regularization terms. The method is broadly applicable to neural networks, dynamical system discovery, basis-function regression, and neural operators. Empirically, it significantly improves generalization performance and parameter efficiency—particularly in low-data regimes—while preserving geometric structure and interpretability.

This paper addresses the characterization of graph homomorphism indistinguishability over structurally restricted graph classes—such as graphs of bounded treewidth, bounded tree depth, and planar graphs—within a unified algebraic framework. Method: It introduces, for the first time, a synthesis of homomorphism tensor embeddings with group representation theory and linear maps between tensor subspaces, integrating homomorphism counting, tensor algebra, and feasibility theory for linear equation systems. Contribution/Results: (1) It provides exact algebraic characterizations of homomorphism indistinguishability over bounded-treewidth, bounded-pathwidth, and bounded-tree-depth graph classes; (2) it establishes its equivalence to the feasibility of specific linear systems (e.g., $Ax = b$); and (3) it unifies the algebraic foundations underlying logical equivalences (e.g., $C^k$, fixed-point logic with counting) and algebraic equivalences (e.g., Weisfeiler–Leman variants, matrix rank conditions), thereby resolving an open problem posed by Dell et al. concerning the characterization of bounded-pathwidth graphs.

The formation mechanisms of internal representations in neural networks remain poorly understood, hindering interpretability of their “black-box” nature. Method: We propose the Canonical Representation Hypothesis (CRH), positing six universal alignment patterns among latent variables, weights, and neuron-wise gradients during training—driving the natural emergence of compact, transformation-invariant representations. CRH is integrated with the Parameter Alignment Hypothesis (PAH) into a unified dual-framework, supported by theoretical analysis of gradient-noise–regularization trade-offs, alignment-aware analysis, power-law modeling, minimal-assumption derivation, and implicit-space geometric characterization. Results: Empirical validation confirms that CRH-compliant representations exhibit compactness and robustness to task-irrelevant transformations; its breakdown triggers power-law alignment, offering a novel theoretical paradigm for generalization and structural emergence in deep learning.

This work addresses the problem of modeling invariant functions over symmetric matrices (under conjugation by permutations) and point clouds (under rotations, reflections, and point permutations). Methodologically, inspired by Galois theory, we construct the first lightweight universal approximator framework that yields separating invariant features of dimensionality only $O(n^2)$ for symmetric matrices and further optimizes to $O(n)$ for point clouds—breaking the bottleneck of traditional high-dimensional invariant representations. Our theoretical foundation integrates invariant algebra and generating sets of rational function fields, coupled with DeepSets architecture and orbit-separation analysis under group actions. Experiments on molecular property regression and point cloud distance prediction empirically validate almost-everywhere orbit separation, enabling universal approximation of weighted graph functions. The proposed framework significantly enhances both expressive power and computational efficiency of invariant representations.

This paper investigates whether deep networks under standard supervised training can autonomously learn unencoded symmetries—such as rotation invariance—from partially observed cyclic group-symmetric data, under realistic class-level symmetry heterogeneity (where some classes exhibit full cyclic transformations while others only subsets). Method: We develop the first neural-kernel symmetry learning theory, grounded in infinite-width NTK analysis and group representation-theoretic Fourier analysis. Contribution/Results: The theory reveals that generalization to unseen symmetries hinges on the “overwhelming dominance” of local data structure over symmetry structure within the kernel-induced feature space, and yields a verifiable signal-to-noise criterion in the frequency domain. Our analysis precisely reproduces empirical failures of MLPs, CNNs, and ViTs on rotationally augmented MNIST subsets, and rigorously proves that conventional supervised training cannot acquire symmetries absent from the architectural prior.

This work addresses the long-standing challenge that tensor trains lack a unique canonical form over arbitrary fields—including finite fields—hindering irreducibility testing and standardization. The paper presents the first construction of a unique canonical form for tensor trains valid over any field, combining algebraic structural analysis with a polynomial-time reduction algorithm. This framework enables direct extraction of the canonical representation, principal indices, and corresponding values from a full tensor. Beyond establishing a theoretical upper bound on compression size, the approach substantially extends the theoretical foundations and applicability of tensor networks beyond real or complex fields, demonstrating their efficacy as a universal formalism across diverse algebraic settings.

This work addresses the lack of a systematic theoretical understanding of the generalization performance of invariance-inducing methods such as canonicalization and group averaging. The authors develop a unified theoretical framework to analyze the generalization error of canonicalization, group averaging, and structurally invariant models through upper bounds on covering numbers, establishing a hierarchy among their generalization capabilities. They provide the first rigorous proof that the effectiveness of canonicalization critically depends on its regularity properties, revealing that Hilbert curve-based canonicalization outperforms lexicographic ordering in point cloud tasks because it ensures polynomial growth of covering numbers, whereas lexicographic ordering leads to exponential growth. This theory underpins state-of-the-art point cloud architectures and is corroborated by empirical validation.

While existing machine learning models can process inputs of arbitrary dimensions—such as graphs or point clouds of varying scales—they lack rigorous theoretical guarantees of universality, as classical universal approximation theorems apply only to fixed-dimensional settings. This work introduces the first systematic definition and verification framework for universality across arbitrary input dimensions. By constructing an infinite-dimensional topological space encompassing all finite-dimensional inputs and their limits, and leveraging symmetry analysis together with the theory of compact families, we reveal fundamental limitations in the cross-dimensional universality of mainstream architectures and propose a concise, effective correction. Building upon this framework, we design a novel model that achieves provable universality over arbitrary dimensions, with formal guarantees rooted in function approximation theory.

This work addresses the lack of theoretical foundations for substructure transferability in graph data by bridging transferable substructures with the intrinsic geometry of graph representation spaces from a functional behavior perspective. It proposes the first Riemannian geometry–based framework for learning intrinsic graph geometry, innovatively introducing neural vector bundles and local coordinate charts to construct the GAUGE pretraining architecture. A Dirichlet loss function is designed to enable explicit modeling of intrinsic graph geometry and quantification of transfer difficulty. The method demonstrates significant performance gains over existing models on zero-shot link prediction and graph isomorphism tasks, validating its expressive power and cross-task transferability.

Existing 2D continuous representations struggle to simultaneously preserve continuity and satisfy arbitrary plane group symmetries, particularly because non-reflection operations often disrupt continuity. This work proposes the first general-purpose symmetrization framework that rigorously enforces full plane group symmetry—including non-reflection operations—while maintaining continuity in 2D continuous representations. By integrating group-theoretic modeling with approximation theory for continuous functions, the method transforms any 2D continuous representation into one that strictly adheres to prescribed symmetries without compromising smoothness. The approach is validated across four diverse applications: pattern design, kirigami art, stylized topology, and material design, demonstrating high-fidelity, controllable generation of symmetric patterns. This study thus achieves, for the first time, full compatibility between general plane group symmetries and continuous 2D representations.