🤖 AI Summary
This study investigates how unsupervised autoencoders learn macroscopic physical variables from microscopic spin configurations of the Ising model. By integrating multiscale coarse-graining analysis, recurrent modeling of dynamical trajectories, and nonequilibrium dynamics, the work reveals—for the first time—the flow-field topology in the representation space of autoencoders, driven by prediction error during training, and identifies two distinct learning regimes: magnetization-dominated and energy-dominated. The authors find that models trained with moderate to high learning rates tend to stall in transitional states, yet learning trajectories across diverse hyperparameter settings share universal topological features. This research establishes an interpretable bridge between unsupervised learning and statistical physics, offering a novel perspective on the physical underpinnings of deep learning representations.
📝 Abstract
We study how unsupervised autoencoders trained on microscopic spin configurations from the Ising model learn macroscopic, theory-relevant variables underlying the data-generating process. Without embedding domain knowledge, we mimic a typical discovery setting: We quantify learning across multiple spatial (coarse-graining) scales and reveal two distinct dynamical regimes controlled by main hyperparameters (model depth, width, and learning rate) -- a magnetization-dominated regime and an energy-dominated regime characterized by trade-offs in their representation quality. The first regime is a transitory state exhibiting dynamical scaling and fluctuations that follow an ordering-to-scale; the second gradually shifts resolution towards smaller scales relevant for the energy representation. Deep models trained at moderate and fast rates become arrested before reaching these regimes. With a novel analysis of recursive-dynamic trajectories, we demonstrate that prediction errors induce flow fields that produce a common trajectory topology across all representation spaces. A dynamical viewpoint of learning is established in which intrinsic properties expose the effects of forced changes in representation during training. We utilize the intuition that learning operates as a process driven far from equilibrium by fluctuations from the training data and optimizer to provide an interpretive basis grounded in both the physical world and the machine models that represent it.