🤖 AI Summary
This work aims to construct a conceptual bridge between statistical physics and deep learning for researchers without a physics background. By recasting statistical physics as a natural extension of probability theory, it systematically elucidates the Boltzmann–Gibbs distribution, Ising models, spin glasses, and phase transition theory, while uncovering their intrinsic connections to Hopfield networks and restricted Boltzmann machines (RBMs). The key insight lies in demonstrating the equivalence between integrating out hidden units in RBMs and the renormalization group transformation, thereby revealing a physically grounded mechanism underlying multilayer deep networks. This framework not only deepens the theoretical understanding of deep learning but also provides a clear physical interpretation of the developmental trajectory of large language models.
📝 Abstract
These lecture notes introduce some topics of classical statistical physics, particularly those that are relevant for neural networks and deep learning. Statistical physics is treated as a branch of probability theory or statistics, with the goal of making concepts such as phase transitions and the renormalization group accessible to readers without prior knowledge of physics. We introduce the Boltzmann-Gibbs distribution and the thermodynamic potentials on a finite configuration space, notably for Ising spins and spin-glass models on a lattice, and then define phase transitions as discontinuities that arise in the limit that the number of lattice points goes to infinity. We further introduce Hopfield networks and Boltzmann machines, which are governed by the same energy function as spin-glass models, and discuss the learning algorithm for restricted Boltzmann machines. In this algorithm hidden neurons are integrated out as in the renormalization group. Finally, modern deep learning is introduced, whose early developments were in part motivated by restricted Boltzmann machines in that they carry many layers of hidden neurons. A description of large language models is given.