A Group Theoretic Analysis of the Symmetries Underlying Base Addition and Their Learnability by Neural Networks

2025-07-14

AI Summary

This work investigates how neural networks achieve fundamental generalization in integer addition by learning the underlying symmetric structure of carry operations, focusing on the algebraic nature of carry functions and their influence on inductive bias. Method: We propose a group-theoretic formalism for carry functions, systematically characterizing symmetry structures of both standard and non-standard carry rules, and define complexity metrics based on group actions (e.g., orbit size, commutativity). Combining theoretical analysis with controlled experiments, we evaluate learning efficiency and generalization across diverse carry structures under matched input representations. Contribution/Results: We demonstrate that minimal neural architectures achieve perfect generalization when input representations align with the carry’s symmetry. Crucially, learning speed and generalization robustness are governed by the algebraic simplicity of the carry’s group action—specifically, faster and more robust generalization arises with abelian groups and low-orbit actions. This work establishes the first unified framework integrating group-theoretic modeling and empirical validation for symmetry-driven generalization in arithmetic tasks.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

A major challenge in the use of neural networks both for modeling human cognitive function and for artificial intelligence is the design of systems with the capacity to efficiently learn functions that support radical generalization. At the roots of this is the capacity to discover and implement symmetry functions. In this paper, we investigate a paradigmatic example of radical generalization through the use of symmetry: base addition. We present a group theoretic analysis of base addition, a fundamental and defining characteristic of which is the carry function -- the transfer of the remainder, when a sum exceeds the base modulus, to the next significant place. Our analysis exposes a range of alternative carry functions for a given base, and we introduce quantitative measures to characterize these. We then exploit differences in carry functions to probe the inductive biases of neural networks in symmetry learning, by training neural networks to carry out base addition using different carries, and comparing efficacy and rate of learning as a function of their structure. We find that even simple neural networks can achieve radical generalization with the right input format and carry function, and that learning speed is closely correlated with carry function structure. We then discuss the relevance this has for cognitive science and machine learning.

Problem

• Analyzing symmetry functions in base addition for neural networks
• Exploring alternative carry functions and their quantitative measures
• Investigating neural network inductive biases in symmetry learning

Innovation

• Group theory analyzes base addition symmetries
• Neural networks learn alternative carry functions
• Input format affects generalization and learning speed