Complete Identification of Deep ReLU Neural Networks by Many-Valued Logic

2026-01-30

AI Summary

This work addresses the complete identification problem for deep ReLU neural networks: given a function, it characterizes all feedforward ReLU network architectures and parameters that implement it. By establishing, for the first time, a full correspondence between ReLU networks and Łukasiewicz many-valued logic, the authors translate networks into logical formulas, apply algebraic rewriting systems together with Chang’s completeness theorem to perform equivalence transformations, and then map the resulting formulas back to neural networks via a novel compositional normal form. Theoretically, they prove that all functionally equivalent ReLU networks can be transformed into one another through a finite set of symmetry operations corresponding to logical axioms. Analogous to Shannon’s theory for Boolean circuits, this framework lays the foundation for structural identification, simplification, and formal analysis of neural networks.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Social Networks and Social Media: Social media analysis through the lenses of networks
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Deep ReLU neural networks admit nontrivial functional symmetries: vastly different architectures and parameters (weights and biases) can realize the same function. We address the complete identification problem -- given a function f, deriving the architecture and parameters of all feedforward ReLU networks giving rise to f. We translate ReLU networks into Lukasiewicz logic formulae, and effect functional equivalent network transformations through algebraic rewrites governed by the logic axioms. A compositional norm form is proposed to facilitate the mapping from Lukasiewicz logic formulae back to ReLU networks. Using Chang's completeness theorem, we show that for every functional equivalence class, all ReLU networks in that class are connected by a finite set of symmetries corresponding to the finite set of axioms of Lukasiewicz logic. This idea is reminiscent of Shannon's seminal work on switching circuit design, where the circuits are translated into Boolean formulae, and synthesis is effected by algebraic rewriting governed by Boolean logic axioms.

Problem

• ReLU neural networks
• functional equivalence
• complete identification
• many-valued logic
• network symmetries

Innovation

• ReLU neural networks
• many-valued logic
• functional equivalence
• algebraic rewriting
• compositional normal form