🤖 AI Summary
This work addresses the limitation of conventional variational autoencoders (VAEs), whose encoders—constrained by the reparameterization trick—struggle to model complex posterior distributions. To overcome this, the authors propose a novel encoder that, for the first time, integrates a diffusion model into the VAE encoding process. They further introduce an alternating training strategy inspired by the Expectation-Maximization (EM) algorithm, which effectively aligns the optimization objectives of the encoder and decoder, thereby ensuring reliable synchronization in the latent space. The proposed approach preserves the simplicity and efficiency of standard diffusion model training while substantially enhancing the model’s capacity to capture complex data distributions and improving reconstruction quality.
📝 Abstract
We construct a new kind of encoder, leveraging the expressive power of diffusion models. In a traditional variational autoencoder, the encoder and decoder jointly negotiate a latent representation of the input. This is made possible by the reparameterization trick, which simplifies training at the cost of restricting the encoder to a simple family of distributions. Replacing this encoder with a diffusion model requires rethinking how the decoder pressure can be transmitted back to the encoder, given that they tend to update their internal estimates of the latent in opposing directions. We solve this problem with an alternating training scheme, inspired by the expectation-maximization algorithm. Our method enables more reliable synchronization between encoder and decoder, while preserving the simple and efficient training objective of standard diffusion models.