Identifiability of Deep Polynomial Neural Networks

2025-06-20

AI Summary

This work addresses the identifiability problem of deep Polynomial Neural Networks (PNNs)—i.e., whether model parameters can be uniquely recovered to ensure interpretability. To fill a critical theoretical gap, we establish the first general and verifiable identifiability criterion for PNNs. Leveraging low-rank tensor decomposition, Kruskal-type uniqueness theorems, and algebraic-geometric techniques, we rigorously prove that PNNs with non-increasing layer widths are identifiable under generic conditions. We fully resolve the long-standing conjecture on the dimension of PNN neural clusters and derive the optimal upper bound on activation counts. Furthermore, we characterize the identifiability boundary for encoder-decoder–structured PNNs, providing explicit, parameter-dependent necessary and sufficient conditions. Collectively, these results furnish a rigorous theoretical foundation for the interpretability of deep polynomial models and deliver practical, computationally tractable verification tools.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Polynomial Neural Networks (PNNs) possess a rich algebraic and geometric structure. However, their identifiability -- a key property for ensuring interpretability -- remains poorly understood. In this work, we present a comprehensive analysis of the identifiability of deep PNNs, including architectures with and without bias terms. Our results reveal an intricate interplay between activation degrees and layer widths in achieving identifiability. As special cases, we show that architectures with non-increasing layer widths are generically identifiable under mild conditions, while encoder-decoder networks are identifiable when the decoder widths do not grow too rapidly. Our proofs are constructive and center on a connection between deep PNNs and low-rank tensor decompositions, and Kruskal-type uniqueness theorems. This yields both generic conditions determined by the architecture, and effective conditions that depend on the network's parameters. We also settle an open conjecture on the expected dimension of PNN's neurovarieties, and provide new bounds on the activation degrees required for it to reach its maximum.

Problem

• Analyzing identifiability of deep Polynomial Neural Networks (PNNs).
• Exploring interplay between activation degrees and layer widths.
• Establishing conditions for generic and effective identifiability.

Innovation

• Analyzes deep PNNs identifiability via algebraic methods
• Links PNNs to tensor decompositions for uniqueness proofs
• Determines identifiability conditions based on network architecture