🤖 AI Summary
This paper investigates the intrinsic structural properties of individual feedforward neural networks from a statistical mechanics perspective, aiming to uncover nontrivial organizational principles—beyond conventional metrics such as loss and accuracy—that govern data fitting, model capacity, generalization, and robustness.
Method: We formulate a Hopfield-like spin-glass model for neural networks and, for the first time, apply replica symmetry breaking (RSB) theory to analyze single network instances. Structural descriptors are defined via inter-replica overlap, yielding computationally tractable, label-free, and training-set-agnostic quantifications of implicit structural complexity solely from network weights.
Contribution/Results: The proposed descriptor effectively discriminates between distinct training states and architectures. Empirically, it demonstrates practical utility in model diagnostics, safety verification, and detection of latent vulnerabilities—providing a theoretically grounded, physics-inspired lens for characterizing neural network structure beyond empirical performance metrics.
📝 Abstract
This work presents a statistical mechanics characterization of neural networks, motivated by the replica symmetry breaking (RSB) phenomenon in spin glasses. A Hopfield-type spin glass model is constructed from a given feedforward neural network (FNN). Overlaps between simulated replica samples serve as a characteristic descriptor of the FNN. The connection between the spin-glass description and commonly studied properties of the FNN -- such as data fitting, capacity, generalization, and robustness -- has been investigated and empirically demonstrated. Unlike prior analytical studies that focus on model ensembles, this method provides a computable descriptor for individual network instances, which reveals nontrivial structural properties that are not captured by conventional metrics such as loss or accuracy. Preliminary results suggests its potential for practical applications such as model inspection, safety verification, and detection of hidden vulnerabilities.