🤖 AI Summary
Posterior collapse in variational autoencoders (VAEs) is commonly attributed to optimization failure, obscuring its fundamental origins. Method: We formulate posterior collapse as a phase transition phenomenon jointly driven by data structure and model hyperparameters, drawing on statistical physics principles; we analyze the stability of the KL divergence between the approximate posterior and prior via variational inference, and identify a critical point where this divergence exhibits discontinuity—marking the phase boundary between collapse and effective latent inference. Contribution/Results: Our framework systematically characterizes the phase transition behavior on both synthetic and real-world datasets, precisely identifying hyperparameter thresholds that prevent collapse. This work establishes a novel theoretical foundation for understanding trainability, representational capacity, and latent-space geometry in deep generative models, offering actionable guidance for VAE design and hyperparameter selection.
📝 Abstract
We investigate the phenomenon of posterior collapse in variational autoencoders (VAEs) from the perspective of statistical physics, and reveal that it constitutes a phase transition governed jointly by data structure and model hyper-parameters. By analyzing the stability of the trivial solution associated with posterior collapse, we identify a critical hyper-parameter threshold. This critical boundary, separating meaningful latent inference from collapse, is characterized by a discontinuity in the KL divergence between the approximate posterior and the prior distribution. We validate this critical behavior on both synthetic and real-world datasets, confirming the existence of a phase transition. Our results demonstrate that posterior collapse is not merely an optimization failure, but rather an emerging phase transition arising from the interplay between data structure and variational constraints. This perspective offers new insights into the trainability and representational capacity of deep generative models.