🤖 AI Summary
This work addresses the amortization gap in variational autoencoders (VAEs), which arises from sharing encoder parameters across all inputs and leads to inaccurate posterior approximations. To mitigate this limitation, the authors propose the Instance-Adaptive Variational Autoencoder (IA-VAE), which employs a hypernetwork to dynamically generate input-specific modulation parameters for the encoder. This enables instance-dependent adaptation of the inference model within a single forward pass, enhancing model expressiveness and parameter efficiency without sacrificing computational tractability. Empirical evaluations on both synthetic data and standard image benchmarks demonstrate that IA-VAE achieves more accurate posterior approximations and higher test-set evidence lower bounds (ELBOs), effectively narrowing the amortization gap.
📝 Abstract
Latent variable models, including variational autoencoders (VAE), remain a central tool in modern deep generative modeling due to their scalability and a well-founded probabilistic formulation. These models rely on amortized variational inference to enable efficient posterior approximation, but this efficiency comes at the cost of a shared parametrization, giving rise to the amortization gap. We propose the instance-adaptive variational autoencoder (IA-VAE), an amortized variational inference framework in which a hypernetwork generates input-dependent modulations of a shared encoder. This enables input-specific adaptation of the inference model while preserving the efficiency of a single forward pass. By leveraging instance-specific parameter modulations, the proposed approach can achieve performance comparable to standard encoders with substantially fewer parameters, indicating a more efficient use of model capacity. Experiments on synthetic data, where the true posterior is known, show that IA-VAE yields more accurate posterior approximations and reduces the amortization gap. Similarly, on standard image benchmarks, IA-VAE consistently improves held-out ELBO over baseline VAEs, with statistically significant gains across multiple runs. These results suggest that increasing the flexibility of the inference parametrization through instance-adaptive modulation is a key factor in mitigating amortization-induced suboptimality in deep generative models.